aboutsummaryrefslogtreecommitdiff
path: root/arch/arm64/mm
AgeCommit message (Collapse)Author
2013-06-14ARM64: mm: HugeTLB support.Steve Capper
Add huge page support to ARM64, different huge page sizes are supported depending on the size of normal pages: PAGE_SIZE is 4KB: 2MB - (pmds) these can be allocated at any time. 1024MB - (puds) usually allocated on bootup with the command line with something like: hugepagesz=1G hugepages=6 PAGE_SIZE is 64KB: 512MB - (pmds) usually allocated on bootup via command line. Signed-off-by: Steve Capper <steve.capper@linaro.org> Acked-by: Catalin Marinas <catalin.marinas@arm.com>
2013-06-14ARM64: mm: Restore memblock limit when map_mem finished.Steve Capper
In paging_init the memblock limit is set to restrict any addresses returned by early_alloc to fit within the initial direct kernel mapping in swapper_pg_dir. This allows map_mem to allocate puds, pmds and ptes from the initial direct kernel mapping. The limit stays low after paging_init() though, meaning any bootmem allocations will be from a restricted subset of memory. Gigabyte huge pages, for instance, are normally allocated from bootmem as their order (18) is too large for the default buddy allocator (MAX_ORDER = 11). This patch restores the memblock limit when map_mem has finished, allowing gigabyte huge pages (and other objects) to be allocated from all of bootmem. Signed-off-by: Steve Capper <steve.capper@linaro.org> Acked-by: Catalin Marinas <catalin.marinas@arm.com>
2013-05-24arm64: Do not report user faults for handled signalsCatalin Marinas
Currently user faults (page, undefined instruction) are always reported even though the user may have a signal handler for them. This patch adds unhandled_signal() check together with printk_ratelimit() for these cases. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2013-05-14arm64: mm: Fix operands of clz in __flush_dcache_allSukanto Ghosh
The format of the lower 32-bits of the 64-bit operand to 'dc cisw' is unchanged from ARMv7 architecture and the upper bits are RES0. This implies that the 'way' field of the operand of 'dc cisw' occupies the bit-positions [31 .. (32-A)]. Due to the use of 64-bit extended operands to 'clz', the existing implementation of __flush_dcache_all is incorrectly placing the 'way' field in the bit-positions [63 .. (64-A)]. Signed-off-by: Sukanto Ghosh <sghosh@apm.com> Tested-by: Anup Patel <anup.patel@linaro.org> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Cc: stable@vger.kernel.org
2013-05-13arm64: debug: clear mdscr_el1 instead of taking the OS lockWill Deacon
During boot, we take the debug OS lock before interrupts are enabled. This is required to prevent clearing of PSTATE.D on the interrupt entry path, which could result in spurious debug exceptions before we've got round to resetting things like the hardware breakpoints registers to a sane state. A problem with this approach is that taking the OS lock prevents an external JTAG debugger from debugging the system, which is especially irritating during boot, where JTAG debugging can be most useful. This patch clears mdscr_el1 rather than taking the lock, clearing the MDE and KDE bits and preventing self-hosted hardware debug exceptions from occurring. Tested-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Cc: stable@vger.kernel.org
2013-05-08Merge tag 'arm64-for-linus' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/cmarinas/linux-aarch64 Pull arm64 update from Catalin Marinas: - Since drivers/irqchip/irq-gic.c no longer has dependencies on arm32 specifics (the 'gic' branch merged), it can be enabled on arm64. - Enable arm64 support for poweroff/restart (for code under drivers/power/reset/). - Fixes (dts file, exception handling, bitops) * tag 'arm64-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/cmarinas/linux-aarch64: arm64: Treat the bitops index argument as an 'int' arm64: Ignore the 'write' ESR flag on cache maintenance faults arm64: dts: fix #address-cells for foundation-v8 arm64: vexpress: Add support for poweroff/restart arm64: Enable support for the ARM GIC interrupt controller
2013-05-08arm64: Ignore the 'write' ESR flag on cache maintenance faultsCatalin Marinas
ESR.WnR bit is always set on data cache maintenance faults even though the page is not required to have write permission. If a translation fault (page not yet mapped) happens for read-only user address range, Linux incorrectly assumes a permission fault. This patch adds the check of the ESR.CM bit during the page fault handling to ignore the 'write' flag. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Reported-by: Tim Northover <Tim.Northover@arm.com> Cc: stable@vger.kernel.org
2013-04-30Merge tag 'arm64-for-linus' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/cmarinas/linux-aarch64 Pull arm64 update from Catalin Marinas: "Main features: - Versatile Express SoC (model) support - DT files and Kconfig entries (there are no arch/arm64/mach-* directories). The bulk of the code has already been moved to drivers/ as part of the ARM SoC clean-up. - Basic multi-cluster support (CPU logical map initialised from the DT) - Simple earlyprintk support for UART 8250/16550 and FastModel console output - Optimised kernel library bitops and string functions. - Automatic initialisation of the irqchip and clocks via DT" * tag 'arm64-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/cmarinas/linux-aarch64: (26 commits) arm64: Use acquire/release semantics instead of explicit DMB arm64: klib: bitops: fix unpredictable stxr usage arm64: vexpress: Enable ARMv8 RTSM model (SoC) support arm64: vexpress: Add dts files for the ARMv8 RTSM models arm64: Survive invalid cpu enable-methods arm64: mm: Correct show_pte behaviour arm64: Fix compat types affecting struct compat_stat arm64: Execute DSB during thread switching for TLB/cache maintenance arm64: compiling issue, need add include/asm/vga.h file arm64: smp: honour #address-size when parsing CPU reg property arm64: Define cmpxchg64 and cmpxchg64_local for outside use arm64: Define readq and writeq for driver module using arm64: Fix task tracing arm64: add explicit symbols to ESR_EL1 decoding arm64: Use irqchip_init() for interrupt controller initialisation arm64: psci: Use the MPIDR values from cpu_logical_map for cpu ids. arm64: klib: Optimised atomic bitops arm64: klib: Optimised string functions arm64: klib: Optimised memory functions arm64: head: match all affinity levels in the pen of the secondaries ...
2013-04-29sparse-vmemmap: specify vmemmap population range in bytesJohannes Weiner
The sparse code, when asking the architecture to populate the vmemmap, specifies the section range as a starting page and a number of pages. This is an awkward interface, because none of the arch-specific code actually thinks of the range in terms of 'struct page' units and always translates it to bytes first. In addition, later patches mix huge page and regular page backing for the vmemmap. For this, they need to call vmemmap_populate_basepages() on sub-section ranges with PAGE_SIZE and PMD_SIZE in mind. But these are not necessarily multiples of the 'struct page' size and so this unit is too coarse. Just translate the section range into bytes once in the generic sparse code, then pass byte ranges down the stack. Signed-off-by: Johannes Weiner <hannes@cmpxchg.org> Cc: Ben Hutchings <ben@decadent.org.uk> Cc: Bernhard Schmidt <Bernhard.Schmidt@lrz.de> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Russell King <rmk@arm.linux.org.uk> Cc: Ingo Molnar <mingo@elte.hu> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: "H. Peter Anvin" <hpa@zytor.com> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org> Cc: "Luck, Tony" <tony.luck@intel.com> Cc: Heiko Carstens <heiko.carstens@de.ibm.com> Acked-by: David S. Miller <davem@davemloft.net> Tested-by: David S. Miller <davem@davemloft.net> Cc: Wu Fengguang <fengguang.wu@intel.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2013-04-29mm/ARM: use common help functions to free reserved pagesJiang Liu
Use common help functions to free reserved pages. Signed-off-by: Jiang Liu <jiang.liu@huawei.com> Cc: Russell King <linux@arm.linux.org.uk> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Will Deacon <will.deacon@arm.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2013-04-25arm64: mm: Correct show_pte behaviourSteve Capper
show_pte makes use of the *_none_or_clear_bad style functions. If a pgd, pud or pmd is identified as being bad, it will then be cleared. As show_pte appears to be called from either the user or kernel fault handlers this side effect can lead to unpredictable behaviour; especially as TLB entries are not invalidated. This patch removes the page table sanitisation from show_pte. If a bad pgd, pud or pmd is encountered it is left unmodified. Signed-off-by: Steve Capper <steve.capper@linaro.org> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2013-03-25ARM64: early_printk: Fix check for CONFIG_ARM64_64K_PAGESBen Hutchings
The 'CONFIG_' prefix is not implicit in IS_ENABLED(). Signed-off-by: Ben Hutchings <ben@decadent.org.uk> Cc: Arnd Bergmann <arnd@arndb.de> Cc: Paul Bolle <pebolle@tiscali.nl> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2013-02-23memory-hotplug: remove memmap of sparse-vmemmapTang Chen
Introduce a new API vmemmap_free() to free and remove vmemmap pagetables. Since pagetable implements are different, each architecture has to provide its own version of vmemmap_free(), just like vmemmap_populate(). Note: vmemmap_free() is not implemented for ia64, ppc, s390, and sparc. [mhocko@suse.cz: fix implicit declaration of remove_pagetable] Signed-off-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com> Signed-off-by: Jianguo Wu <wujianguo@huawei.com> Signed-off-by: Wen Congyang <wency@cn.fujitsu.com> Signed-off-by: Tang Chen <tangchen@cn.fujitsu.com> Cc: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> Cc: Jiang Liu <jiang.liu@huawei.com> Cc: Kamezawa Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> Cc: Lai Jiangshan <laijs@cn.fujitsu.com> Cc: Wu Jianguo <wujianguo@huawei.com> Cc: Ingo Molnar <mingo@elte.hu> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: "H. Peter Anvin" <hpa@zytor.com> Signed-off-by: Michal Hocko <mhocko@suse.cz> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2013-01-22arm64: Add simple earlyprintk supportCatalin Marinas
This patch adds support for "earlyprintk=" parameter on the kernel command line. The format is: earlyprintk=<name>[,<addr>][,<options>] where <name> is the name of the (UART) device, e.g. "pl011", <addr> is the I/O address. The <options> aren't currently used. The mapping of the earlyprintk device is done very early during kernel boot and there are restrictions on which functions it can call. A special early_io_map() function is added which creates the mapping from the pre-defined EARLY_IOBASE to the device I/O address passed via the kernel parameter. The pgd entry corresponding to EARLY_IOBASE is pre-populated in head.S during kernel boot. Only PL011 is currently supported and it is assumed that the interface is already initialised by the boot loader before the kernel is started. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Arnd Bergmann <arnd@arndb.de>
2012-11-23arm64: Convert empty flush_cache_{mm,page} functions to static inlineCatalin Marinas
These functions are empty, just make them static inline in the header. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2012-11-14arm64: mm: update max_dma32 before calculating size of NORMAL zoneWill Deacon
Commit f483a853b0b9 ("arm64: mm: fix booting on systems with no memory below 4GB") sets max_dma32 to the minimum of the maximum pfn and MAX_DMA32_PFN. This value is later used as the base of the NORMAL zone, which is incorrect when MAX_DMA32_PFN is below the minimum pfn (i.e. all memory is above 4GB). This patch fixes the problem by ensuring that max_dma32 is always set to the end of the DMA32 zone. Signed-off-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2012-11-14arm64: Make the user fault reporting more specificCatalin Marinas
For user space faults the kernel reports "unhandled page fault" and it gives the ESR value. With this patch the error message looked up in the fault info array to give a better description. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2012-11-08arm64: mm: fix booting on systems with no memory below 4GBWill Deacon
Booting on a system with all of its memory above the 4GB boundary breaks for two reasons: (1) We still try to create a non-empty DMA32 zone (2) no-bootmem limits allocations to 0xffffffff This patch fixes these issues for ARM64. Tested-by: Marc Zyngier <marc.zyngier@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2012-10-08arm64: Call swiotlb_init() instead of swiotlb_init_with_default_size()Catalin Marinas
Following commit 74838b7 (swiotlb: add the late swiotlb initialization function with iotlb memory) the swiotlb_init_with_default_size() is a static function. This patch changes the arm64 code to call swiotlb_init() instead and use the default size of 64MB. It is assumed that AArch64 platforms have enough RAM to afford the pre-allocated swiotlb memory. It also removes the #ifdef around this call since CONFIG_SWIOTLB is always enabled. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2012-09-24arm64: Do not set the SMP/nAMP processor bitCatalin Marinas
If such bit exists on a given CPU, it must be set by the firmware or boot-loader prior to starting the kernel (see Documentation/arm64/booting.txt). Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2012-09-17arm64: Build infrastructureCatalin Marinas
This patch adds Makefile and Kconfig files required for building an AArch64 kernel. Signed-off-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Tony Lindgren <tony@atomide.com> Acked-by: Nicolas Pitre <nico@linaro.org> Acked-by: Olof Johansson <olof@lixom.net> Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com> Acked-by: Arnd Bergmann <arnd@arndb.de>
2012-09-17arm64: DMA mapping APICatalin Marinas
This patch adds support for the DMA mapping API. It uses dma_map_ops for flexibility and it currently supports swiotlb. This patch could be simplified further if the DMA accesses are coherent (not mandated by the architecture) or if corresponding hooks are placed in the generic swiotlb code to deal with cache maintenance. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Tony Lindgren <tony@atomide.com> Acked-by: Nicolas Pitre <nico@linaro.org> Acked-by: Olof Johansson <olof@lixom.net> Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com> Acked-by: Arnd Bergmann <arnd@arndb.de>
2012-09-17arm64: Device specific operationsCatalin Marinas
This patch adds several definitions for device communication, including I/O accessors and ioremap(). The __raw_* accessors are implemented as inline asm to avoid compiler generation of post-indexed accesses (less efficient to emulate in a virtualised environment). Signed-off-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Arnd Bergmann <arnd@arndb.de> Acked-by: Tony Lindgren <tony@atomide.com> Acked-by: Nicolas Pitre <nico@linaro.org> Acked-by: Olof Johansson <olof@lixom.net> Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
2012-09-17arm64: TLB maintenance functionalityCatalin Marinas
This patch adds the TLB maintenance functions. There is no distinction made between the I and D TLBs. TLB maintenance operations are automatically broadcast between CPUs in hardware. The inner-shareable operations are always present, even on UP systems. NOTE: Large part of this patch to be dropped once Peter Z's generic mmu_gather patches are merged. Signed-off-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Tony Lindgren <tony@atomide.com> Acked-by: Nicolas Pitre <nico@linaro.org> Acked-by: Olof Johansson <olof@lixom.net> Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com> Acked-by: Arnd Bergmann <arnd@arndb.de>
2012-09-17arm64: Cache maintenance routinesCatalin Marinas
The patch adds functionality required for cache maintenance. The AArch64 architecture mandates non-aliasing VIPT or PIPT D-cache and VIPT (may have aliases) or ASID-tagged VIVT I-cache. Cache maintenance operations are automatically broadcast in hardware between CPUs. Signed-off-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Tony Lindgren <tony@atomide.com> Acked-by: Nicolas Pitre <nico@linaro.org> Acked-by: Olof Johansson <olof@lixom.net> Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com> Acked-by: Arnd Bergmann <arnd@arndb.de>
2012-09-17arm64: CPU supportCatalin Marinas
This patch adds AArch64 CPU specific functionality. It assumes that the implementation is generic to AArch64 and does not require specific identification. Different CPU implementations may require the setting of various ACTLR_EL1 bits but such information is not currently available and it should ideally be pushed to firmware. Signed-off-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Tony Lindgren <tony@atomide.com> Acked-by: Nicolas Pitre <nico@linaro.org> Acked-by: Olof Johansson <olof@lixom.net> Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
2012-09-17arm64: Process managementCatalin Marinas
The patch adds support for thread creation and context switching. The context switching CPU specific code is introduced with the CPU support patch (part of the arch/arm64/mm/proc.S file). AArch64 supports ASID-tagged TLBs and the ASID can be either 8 or 16-bit wide (detectable via the ID_AA64AFR0_EL1 register). Signed-off-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Tony Lindgren <tony@atomide.com> Acked-by: Nicolas Pitre <nico@linaro.org> Acked-by: Olof Johansson <olof@lixom.net> Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com> Acked-by: Arnd Bergmann <arnd@arndb.de>
2012-09-17arm64: MMU fault handling and page table managementCatalin Marinas
This patch adds support for the handling of the MMU faults (exception entry code introduced by a previous patch) and page table management. The user translation table is pointed to by TTBR0 and the kernel one (swapper_pg_dir) by TTBR1. There is no translation information shared or address space overlapping between user and kernel page tables. Signed-off-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Tony Lindgren <tony@atomide.com> Acked-by: Nicolas Pitre <nico@linaro.org> Acked-by: Olof Johansson <olof@lixom.net> Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com> Acked-by: Arnd Bergmann <arnd@arndb.de>
2012-09-17arm64: MMU initialisationCatalin Marinas
This patch contains the initialisation of the memory blocks, MMU attributes and the memory map. Only five memory types are defined: Device nGnRnE (equivalent to Strongly Ordered), Device nGnRE (classic Device memory), Device GRE, Normal Non-cacheable and Normal Cacheable. Cache policies are supported via the memory attributes register (MAIR_EL1) and only affect the Normal Cacheable mappings. This patch also adds the SPARSEMEM_VMEMMAP initialisation. Signed-off-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Tony Lindgren <tony@atomide.com> Acked-by: Nicolas Pitre <nico@linaro.org> Acked-by: Olof Johansson <olof@lixom.net> Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com> Acked-by: Arnd Bergmann <arnd@arndb.de>
2012-09-17arm64: Assembly macros and definitionsCatalin Marinas
This patch introduces several assembly macros and definitions used in the .S files across arch/arm64/ like IRQ disabling/enabling, together with asm-offsets.c. Signed-off-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Tony Lindgren <tony@atomide.com> Acked-by: Arnd Bergmann <arnd@arndb.de> Acked-by: Nicolas Pitre <nico@linaro.org> Acked-by: Olof Johansson <olof@lixom.net> Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>