path: root/Makefile
AgeCommit message (Collapse)Author
2010-09-24Change PREFIX and EXEDIR to be set with ?=Eric B Munson
This patch alters the Makefile to use ?= to set the EXEDIR and the PREFIX which will allow easier modification of these values during scripted builds. Signed-off-by: Eric B Munson <emunson@mgebm.net>
2010-03-30Disable executable stacks for library and helpers V4Eric B Munson
Currently the library and all the helpers are built with their stack mappings markes executable. This could be a security issue and shouldn't be allowed unless necessary. This patch disables executable stacks for the library and the helper programs. Signed-off-by: Eric B Munson <ebmunson@us.ibm.com>
2009-12-20Change INSTALL_STAT target to INSTALL_SCRIPTEric B Munson
This target will be used for scripts that do not need to be compiled. This patch also moves the new huge_page_setup_helper.py into its own group which is not installed by default. Signed-off-by: Eric B Munson <ebmunson@us.ibm.com>
2009-12-18libhugetlbfs huge page setup helperJarod Wilson
A common complaint we (Red Hat) get from customers is that setting up huge pages for use with their java or database applications is complex. While hugeadm makes this easier than it has been in the past, some customers want brain-dead simple, fire and forget setup. The attached patch adds a python script which attempts to implement that, asking the user only three questions (how much memory and what users and group should be able to use it), which should get them up and going. Signed-off-by: Jarod Wilson <jarod@redhat.com> Signed-off-by: Eric B Munson <ebmunson@us.ibm.com>
2009-07-22Adjust tool compiler to build 64 bit tools if availableEric B Munson
The make process currently builds all lib huge tools as 32 bit binaries regardless of the build environment. This causes a problem on ppc64 when attempting to deal with 16GB pages as their size is larger than a 32 bit long long. This patch updates the tools build to use the 64 bit compiler whenever it is available. Signed-off-by: Eric B Munson <ebmunson@us.ibm.com>
2009-07-22Add PerfCollector perl module to install listEric B Munson
Signed-off-by: Eric B Munson <ebmunson@us.ibm.com>
2009-06-23Add perf counter scriptEric B Munson
This patch adds the cpupcstat script that can be used to measure the DTLB miss rate while executing a target program. It is based on and borrows heavily from the VMRegress tool written by Mel Gorman <mel@csn.ul.ie>. cpupcstat currently uses oprofile to collect DTLB miss rate data, though the data collection is modular enough that it should be simple to add another collection method. The number of new oprofile samples since the last report and the sample rate for this interval. Hopefully this will be useful to identify workloads that would benefit from using hugepages. Thanks to Mel Gorman for several contributions the cpupcstat tool and for letting me cannibalize oprofile_start.sh and oprofile_map_events.pl from vmregress. Signed-off-by: Eric B Munson <ebmunson@us.ibm.com>
2009-01-30Rework test suite for multiple huge page size testingAdam Litke
Now that libhugetlbfs can work with multiple huge page sizes, testing this support has become a priority. The following patch enables automatic testing of page sizes that are mounted and have at least one page allocated. Care is taken to assure that only valid combinations are tested. For example, 32bit tests are not run with 16GB pages. Following the run, a summary of all page sizes tested is printed. The following is some example output of the new script: ********** TEST SUMMARY * 64K 16M 16G * 32-bit 64-bit 32-bit 64-bit 32-bit 64-bit * Total testcases: 86 89 86 89 0 89 * Skipped: 20 0 20 0 0 0 * PASS: 59 75 62 85 0 49 * FAIL: 5 6 1 1 0 29 * Killed by signal: 0 0 0 0 0 0 * Bad configuration: 2 2 3 3 0 10 * Expected FAIL: 0 0 0 0 0 0 * Unexpected PASS: 0 0 0 0 0 0 * Strange test result: 0 6 0 0 0 1 ********** Script programming language conversion alert: This patch rewrites run_tests.sh in python. I already anticipate the "why change languages" comments so I come prepared with justification for the conversion. Our test harness has extended beyond simply executing a list of test cases and dumping the output to stdout. The data set for the test summary is now three-dimensional. It is indexed by result type (total tests, total passed, etc), word size, and now page size. The simple arrays in bash are not up to the task. As the number of tests that are run increases, so does the challenge of presenting the results in a meaningful, easy to digest format. Shell scripts lack the output formatting constructs that are present in languages such as Python and that make flexible output formatting possible. For these reasons (and the guarantee that the test harness will need to get even more sophisticated in the future), I made the inevitable decision to cut over to Python now. Signed-off-by: Eric B Munson <ebmunson@us.ibm.com> Signed-off-by: Adam Litke <agl@us.ibm.com>
2008-11-10Add get_hugepage_region() as a malloc-like interface that uses hugepagesMel Gorman
The get_huge_pages() API is a close-to-kernel interface for the direct allocation of hugepages. This forces the caller to deal with alignment and fallback to base pages where suitable. For the casual user of hugepages that does not care for such things, this patch adds get_hugepage_region(). It allocates regions of memory that are backed by hugepages where possible but callers are not required to align their length and can request fallback to base pages. Signed-off-by: Mel Gorman <mel@csn.ul.ie> Acked-by: Adam Litke <agl@us.ibm.com> Acked-by: Eric B Munson <ebmunson@us.ibm.com> Signed-off-by: Eric B Munson <ebmunson@us.ibm.com>
2008-10-30hugectl: fix up Makefile to correctly pass LIB32/LIB64 to hugectlAndy Whitcroft
It seems we are passing the LIB32/LIB64 defines to the link phase and not the compile phase for hugectl. This means that it will not set the library path correctly. Signed-off-by: Andy Whitcroft <apw@shadowen.org> Acked-by: Adam Litke <agl@us.ibm.com> Signed-off-by: Eric B Munson <ebmunson@us.ibm.com>
2008-10-27Handle the case where lib64 and lib32 resolve to the same placeMel Gorman
On recent 64-bit Debian systems, /usr/lib64 is a symbolic link to /usr/lib and 64-bit libraries are installed to /usr/lib/$ARCH-linux-gnu. In this case, the 64-bit install of libhugetlbfs overwrites the 32-bit version of the library. It's different in Ubuntu where lib can be the 64-bit directory, lib32 the 32-bit one and lib64 a symbolic link to lib. When installing with PREFIX=/usr, this leads to all sorts of hilarity and the bizarre situation where the 64-bit install overwrites the 32-bit install. How each distro arranges 32/64 bit libraries in /usr varies considerably so rather than trying to figure out what to do, inform the user when the library installation paths are going to the same place and allow them to manually specify the correct library path names within the prefix. Signed-off-by: Mel Gorman <mel@csn.ul.ie> Signed-off-by: Eric B Munson <ebmunson@us.ibm.com>
2008-10-21stop exporting kernel detectionAndy Whitcroft
Signed-off-by: Andy Whitcroft <apw@shadowen.org> Acked-by: Adam Litke <agl@us.ibm.com> Acked-by: Mel Gorman <mel@csn.ul.ie>
2008-10-21stop exposing pool configuration in the main libraryAndy Whitcroft
Signed-off-by: Andy Whitcroft <apw@shadowen.org> Acked-by: Adam Litke <agl@us.ibm.com> Acked-by: Mel Gorman <mel@csn.ul.ie>
2008-10-21add a new private utilities libraryAndy Whitcroft
Add a new private utilities library consisting of various useful helpers normally hidden within libhugetlbfs. Extend the library local marker idiom to include a private utilities marker __pu_. When we build libhugetlbfs these are forced local when building libhugetlbfs_privutils these are the only routines exported. This makes it very hard for the two libraries to interfere with each other when both are linked to the same binary, which is particularly important when testing the library; we must test the real one. Signed-off-by: Andy Whitcroft <apw@shadowen.org> Acked-by: Mel Gorman <mel@csn.ul.ie> Acked-by: Adam Litke <agl@us.ibm.com>
2008-10-17Allow individual installation of manual pages, utilities or library with ↵Mel Gorman
linker scripts Currently the install target installs the library, utilities and the manual pages. In the situation where the user has downloaded libhugetlbfs for the utilities, it is relatively difficult to build utilities that use the main system paths, but do not override the distribution-supported version of libhugetlbfs. This patch allows a user to do something like $ make PREFIX=/usr $ make install-bin $ make install-man and get the utilities and manual pages using the distribution-supported library. The install target still installs everything and should be functionally equivilant. The only difference is a clarification in output that INSTALL32 is really INSTALL-LIB32 Signed-off-by: Mel Gorman <mel@csn.ul.ie> Signed-off-by: Eric B Munson <ebmunson@us.ibm.com>
2008-10-16Fixing install locations for packagingEric B Munson
The FHS requires that man pages be installed to /usr/share/man* so this patch adjusts the install target for the man pages to $DESTDIR/$PREFIX/share/man and it makes the symlink for free_hugepages use a relative path rather than hard. Signed-off-by: Eric B Munson <ebmunson@us.ibm.com>
2008-10-15This patch creates the man1 and man3 directories if they do not exist.Eric B Munson
Signed-off-by: Mel Gorman <mel@csn.ul.ie> Signed-off-by: Eric B Munson <ebmunson@us.ibm.com>
2008-10-15On PowerPC, always specify '-m32' for 32bit compiles. We don't know2.1-pre2Eric B Munson
whether the compiler defaults to creating 32bit or 64bit binaries so it's best to play it safe and be explicit. Signed-off-by: Adam Litke <agl@us.ibm.com> Signed-off-by: Eric B Munson <ebmunson@us.ibm.com>
2008-10-14pagesize: add utility to list page sizesAndy Whitcroft
Add a new utility to list page sizes supported on the system. Signed-off-by: Andy Whitcroft <apw@shadowen.org> Acked-by: Mel Gorman <mel@csn.ul.ie> Acked-by: Adam Litke <agl@us.ibm.com>
2008-10-14hugeadm: initial basic frameworkAndy Whitcroft
Add a very basic framework for hugeadm, the administrator pool configuration helper. Signed-off-by: Andy Whitcroft <apw@shadowen.org> Acked-by: Mel Gorman <mel@csn.ul.ie> Acked-by: Adam Litke <agl@us.ibm.com>
2008-10-14build: allow utilities to consist of more than one fileAndy Whitcroft
Allow a utility control which source files it is made up of. This allows us to reuse some of the utility functions from within libhugetlbfs directly without linking against the library. Signed-off-by: Andy Whitcroft <apw@shadowen.org> Acked-by: Mel Gorman <mel@csn.ul.ie> Acked-by: Adam Litke <agl@us.ibm.com>
2008-10-13Add a manual page for hugeadmMel Gorman
This patch adds a basic manual page for the pagesize utility. Signed-off-by: Mel Gorman <mel@csn.ul.ie> Signed-off-by: Eric B Munson <ebmunson@us.ibm.com>
2008-10-13Add a manual page for pagesizeMel Gorman
This patch adds a basic manual page for the pagesize utility. Signed-off-by: Mel Gorman <mel@csn.ul.ie> Signed-off-by: Eric B Munson <ebmunson@us.ibm.com>
2008-10-13Add a manual page for getpagesizes()Mel Gorman
This patch adds a basic manual page for getpagesizes(). Signed-off-by: Mel Gorman <mel@csn.ul.ie> Signed-off-by: Eric B Munson <ebmunson@us.ibm.com>
2008-10-13Add a manual page for gethugepagesizes()Mel Gorman
This patch adds a basic manual page for gethugepagesizes(). Signed-off-by: Mel Gorman <mel@csn.ul.ie> Signed-off-by: Eric B Munson <ebmunson@us.ibm.com>
2008-10-09Add a manual page for get_huge_pages() and free_huge_pages()Mel Gorman
This patch adds a basic manual page for get_huge_pages and free_huge_pages(). Signed-off-by: Mel Gorman <mel@csn.ul.ie> Signed-off-by: Eric B Munson <ebmunson@us.ibm.com>
2008-10-09Add a manual page for hugeeditMel Gorman
This patch adds a basic manual page for hugeedit. Signed-off-by: Mel Gorman <mel@csn.ul.ie> Signed-off-by: Eric B Munson <ebmunson@us.ibm.com>
2008-10-09Add a manual page for hugectlMel Gorman
This patch adds a basic manual page for hugectl. Signed-off-by: Mel Gorman <mel@csn.ul.ie> Signed-off-by: Eric B Munson <ebmunson@us.ibm.com>
2008-10-09Add a manual page for libhugetlbfsMel Gorman
This patch adds a basic manual page for libhugetlbfs. It points the user to the HOWTO for detailed instructions and is intended as a brief synopisis for the environment variables. Signed-off-by: Mel Gorman <mel@csn.ul.ie> Signed-off-by: Eric B Munson <ebmunson@us.ibm.com>
2008-10-09must compile explicitly for 32 bitJon Tollefson
In order to compile the 32 bit versions of the test programs correctly you must explicitly specify 32 bit to gcc. This is a fix for some distros that compile 64 bit by default. The switch should be backwards compatible. Signed-off-by: Jon Tollefson <kniht@linux.vnet.ibm.com> Signed-off-by: Eric B Munson <ebmunson@us.ibm.com>
2008-10-02Fix header installation to use DESTDIRMel Gorman
The headers are not being installed to the correct path when DESTDIR is used. This is awkward when building libhugetlbfs for distribution packages, particularly Debian. Signed-off-by: Mel Gorman <mel@csn.ul.ie> Signed-off-by: Eric B Munson <ebmunson@us.ibm.com>
2008-09-04Merge branch 'work/kernel-versions' into work/multiple-huge-page-sizesAdam Litke
2008-08-27hugectl: add support for requesting which library to useAndy Whitcroft
Add support for requesting a specific library set for preload. This adds the --library-path option. If this option points to a directory containing a libhugetlbfs library it is used, else it is assumed to be a library prefix and both the 32 bit and 64 bit library directories are added (where available). By default the specific libraries installed with the version of hugectl are used. Signed-off-by: Andy Whitcroft <apw@shadowen.org> Acked-by: Eric B Munson <ebmunson@us.ibm.com> Acked-by: Mel Gorman <mel@csn.ul.ie>
2008-08-21Fix install target for libhuge header filesEric Munson
When install is called with a single source file it treats the second argument as a path ending in a file name. So when install is called with a single header (hugetlbfs.h) to go into /path/to/install/include, include becomes the file name for hugetlbfs.h. This patch changes the call to install to force the destination to be treated as a directory and install the listed source file(s) in that directory. Singed-off-by: Eric B Munson <ebmunson@us.ibm.com>
2008-08-14Allow shmget() to be overridden to add the SHM_HUGETLB flagMel Gorman
There are applications that are not hugepage-aware but use shared memory such as the postgres database. It is possible that the SHM_HUGETLB flag could be added to such an application via libhugetlbfs and overriding shmget() to add the SHM_HUGETLB to the flags. This patch enables libhugetlbfs to add the SHM_HUGETLB when a HUGETLB_SHM environment variable is set to "yes". It will work whether the application is linked to libhugetlbfs or loaded via LD_PRELOAD. Signed-off-by: Mel Gorman <mel@csn.ul.ie> Acked-by: Adam Litke <agl@us.ibm.com> Signed-off-by: Eric Munson <ebmunson@us.ibm.com>
2008-08-05[RFC] Use the kernel version number to identify kernel functionality V2Adam Litke
Historically, libhugetlbs has relied on kernel features that either: have been known to exist in all supported kernel versions, or are easily detected. As of kernel version 2.6.27-rc1, a new crucial feature has been added that is not possible to reliably detect. Huge page mappings created with the MAP_PRIVATE flag will have huge pages reserved up-front. With private reservations in effect, it is safe to allow demand-faulting of the HUGETLB_MORECORE heap which can lead to dramatic performance improvements on NUMA systems. This is only safe behavior in the presence of private reservations. The only way to identify that a kernel has private reservations support is to examine the kernel version to see if it is more recent than when the feature appeared. I am well aware of the drawbacks of using the kernel version to affect library behavior but I don't see any alternative. I would suggest that the kernel version should be used only in cases when there is no alternative. How it works ============ Kernels are assumed to have a mandatory base version x.y.z (eg. 2.6.17) and one optional modifier: a post version (stable tree x.y.z.q) or a pre version (x.y.z-{preN|rcN}). All other version appendices (such as -mmN) are ignored. The following ordering rules apply: x.y.z-rc(N) < x.y.z-rc(N+1) < x.y.z < x.y.z.(N) < x.y.z.(N+1) When libhugetlbfs initializes, the running kernel version is probed using uname. A list of feature definitions is scanned and those with a minimum kernel version have that version compared to the runninng kernel. If the running kernel is found to be equal to or greater than the minimum required kernel version, a bit in a feature mask is set to indicate the presence of the feature. A feature can be later checked for by using a simple function that checks the bitmask. Changes since V1 (Thanks Andy Whitcroft and Mel Gorman): - Fixed feature_mask handling - Readability improvements
2008-07-31build: remove redundant hugectl stanzaAndy Whitcroft
We have a hugectl specific stanza, but now we use a generic stanza to build the tools into obj/. Remove the redundant stanza. Signed-off-by: Andy Whitcroft <apw@shadowen.org> Signed-off-by: Nishanth Aravamudan <nacc@us.ibm.com>
2008-07-28Add basic s390x supportNishanth Aravamudan
Enable the building of the library on s390 and specify the behavior of the icache-hygiene test. The testsuite runs relatively clean. Only one major issue and I've reported it upstream. Signed-off-by: Nishanth Aravamudan <nacc@us.ibm.com> Acked-by: Adam Litke <agl@us.ibm.com>
2008-07-24tools: hugeedit: Modify binaries to set default remapping behaviorAdam Litke
The recently added relinking method makes it possible to choose from multiple ways to remap segments at run-time via the HUGTLB_ELFMAP environment variable. If nothing is specified in that variable, then no remapping will occur. Sometimes it is desirable to set a default remapping mode so that HUGETLB_ELFMAP does not need to be set. This semantic applied to the original remapping algorithm. This patch adds a utility that can modify the PF_LINUX_HUGETLB segment flags in the ELF haeder of a binary to control which segments will be remapped by default. The tool will also display this information. [Plus fix for argc comparison by Andy W.] Signed-off-by: Adam Litke <agl@us.ibm.com> Tested-by: Andy Whitcroft <apw@shadowen.org> Acked-by: Andy Whitcroft <apw@shadowen.org> Signed-off-by: Nishanth Aravamudan <nacc@us.ibm.com>
2008-07-24build: Cleanup tools buildAdam Litke
To properly support building and installing tools (hugectl, hugeedit, hugecfg, etc), some changes to the initial method of building hugectl are required. This patch creates wildcard rules that can be used for building either 32 or 64 bit versions of all tools. Additionally, a native wordsize is selected for each architecture so that only one binary (of the desired default word size) is built and installed. Signed-off-by: Adam Litke <agl@us.ibm.com> Acked-by: Andy Whitcroft <apw@shadowen.org> Tested-by: Andy Whitcroft <apw@shadowen.org> Signed-off-by: Nishanth Aravamudan <nacc@us.ibm.com>
2008-07-02Provide a direct allocator API for huge pagesMel Gorman
HUGETLB_MORECORE currently exists to allow glibc to back malloc() with large pages instead of small pages. However, not all applications use glibc malloc() nor is it always desirable to back malloc() with huge pages. There exists a requirement that a hugepage-aware application be able to allocate hugepages directly. Currently, each application is expected to discover the filesystem themselves, mmap() the file and other house-keeping tasks. libhugetlbfs already implements much of this complex logic internally. This patch exposes a simple API for the allocation and freeing of regions backed by hugepages. The implementation is a little over-simplistic but can be optimised later if and when applications perceive its performance to be a bottleneck. The API itself should not need to change as a multi-page aware API would be an additional rather than a replacement interface. Signed-off-by: Mel Gorman <mel@csn.ul.ie> Acked-by: David Gibson <david@gibson.dropbear.id.au> Signed-off-by: Nishanth Aravamudan <nacc@us.ibm.com>
2008-07-01Add hugectl to MakefileEric B Munson
This patch adds a build and install target for the libhugetlbfs front end, hugectl. Signed-off-by: Eric Munson <ebmunson@us.ibm.com> Signed-off-by: Nishanth Aravamudan <nacc@us.ibm.com>
2008-05-07Add plt_extrasz for PPC32Eric B Munson
Currently segment remapping on powerpc32 fails to copy the PLT section. This is because, like ppc64, the PLT is NOBITS so it is missed by the PROGBITS copy. This patch adds the elf32ppclinux.c file that defines the plt_extrasz function to compute the PLT size to ensure that it is copied. The only difference from the 64 bit version is the size of PLT[0] is 72 bytes on PPC32 as opposed to 24 bytes on PPC64. Signed-off-by: Eric Munson <ebmunson@us.ibm.com> Signed-off-by: Nishanth Aravamudan <nacc@us.ibm.com>
2008-04-15Skip elflink calls in setup_libhugetlbfs on IA64/sparc64Eric B Munson
Building on IA64 and sparc64 currently fails because elflink is not supported. This patch sets up a NO_ELFLINK define in the appropriate sections of the Makefile and a check in setup_libhugetlbfs that will skip the elflink calls on IA64 and sparc64. Signed-off-by: Eric Munson <ebmunson@us.ibm.com> Signed-off-by: Nishanth Aravamudan <nacc@us.ibm.com>
2008-03-07elflink: Add powerpc64-specific PLT size detectionAdam Litke
PowerPC 64-bit binaries place their plt in the uninitialized (normally zeroed) part of the data segment. This placement requires extra copying to be performed in order to preserve symbol lookups that have already been completed by the dynamic linker. Determining the number of additional bytes to copy is platform-specific. Therefore, we use a new mechanism to specify an implementation for elf64ppc. The build system has been modified to notice $ELF{32|64}.c files and, when found, build them into the library. This implementation replaces the somewhat hand-wavy linker script approach we used in the past to work around this discrepancy. NOTE: This functionality was originally part of the script-less relinking patch series. Since it is a good stand-alone fix I have broken it out and am pushing it on its own merit. The mechanism for supplying arch-specific functions has been changed based on a suggestion from David Gibson. Signed-off-by: Adam Litke <agl@us.ibm.com> Signed-off-by: Nishanth Aravamudan <nacc@us.ibm.com>
2008-03-07build: Rename .S filesAdam Litke
Due to the way our multi-target Make rules work, it is not possible for two files to exist where the only difference in name is the file extension. For example: if foo.c and foo.S existed, the build system would try and build foo.c into foo.o and then build foo.S into foo.o. Oops. Rather than changing the build system to remove this restriction (which I deem is non-trivial), prefix the .S files in a sensible manner so that .c files can be added with the former .S basenames. Since the .S files each contain an assembly stub to make a system call, it seems sensible to prefix these files with 'sys-'. Signed-off-by: Adam Litke <agl@us.ibm.com> Signed-off-by: Nishanth Aravamudan <nacc@us.ibm.com>
2008-02-28libhugetlbfs: consolidate to one constructorNishanth Aravamudan
Use one constructor to control the constructor order for libhugetlbfs. Currently, the constructors are run in the order their containing object files are linked in to libhugetlbfs.so. This is fragile as new features are added. Instead, have one constructor that calls the others (which are now no longer actually constructors). Signed-off-by: Nishanth Aravamudan <nacc@us.ibm.com> Acked-by: Andrew Hastings <abh@cray.com> Acked-by: David Gibson <david@gibson.dropbear.id.au>
2007-08-01build tests before installing themSteve Fox
build: Force tests to be built before we run install-tests Signed-off-by: Steve Fox <drfickle@linux.vnet.ibm.com> Signed-off-by: Nishanth Aravamudan <nacc@us.ibm.com>
2007-07-16Makefile: fix-up recent cleanupNishanth Aravamudan
$@ in the libhugetlbfs.so lines refers to the matching symbol, which in this case is "obj32/libhugetlbfs.so" or "obj64/libhugetlbfs.so". This breaks just about everything when the soname uses $@ directly. Instead use $(notdir $@) to extract only the file part. Signed-off-by: Nishanth Aravamudan <nacc@us.ibm.com>
2007-07-16touchup libhugetlbfs Makefile1.2-pre1Mike Frysinger
Update the Makefile to: - respect env CFLAGS/LDFLAGS/CPPFLAGS - add proper SONAME to shared libs * quoting Mike: "so to answer your question simply, at this very moment in time, because of the way ld behaves (it will guess a SONAME if you dont encode one), linking with -Wl,-soname gains you nothing except it makes some automatic ELF QA checkers not warn. down the road, it will be needed if you start using ABI numbers if the filename. [Ed: e.g. libhugetlbfs.so.1.0, libhugetlbfs.so.1]" - make "install" target depend on "libs", not "all" otherwise test gets forced ... this allows package maintainers to split the test/install steps up properly in their distribution Signed-off-by: Mike Frysinger <vapier@gentoo.org> Signed-off-by: Nishanth Aravamudan <nacc@us.ibm.com>