Creating a Package
-
Before you start creating packages, you need to set the
EDITOR
variable, where you specify your preferred editor while using Spack. For example, to choosevim
as an editor, you type:export EDITOR='/usr/bin/vim'
-
For more details on creating packages, visit Spack Documentation.
Adding a Package Repository
-
In this tutorial, you will learn how to re-create some already created packages. Therefore, you will add a package repository
myproject
, which you have created before, to avoid modifying Spack installation with the package you are creating. To add themyproject
repository, type:spack repo add $SPACK_ROOT/var/spack/repos/myproject
Creating the Package File
-
As mentioned before, Spack comes with thousands of build-in packages. Suppose you want to create a new package, Spack allows users to create new packages in their own package repositories. To create a new package, you type:
spack create <url>
-
url
is the link that contains the software repository. -
The
spack create
command builds a new package from a template by taking the location of the package source code and using it to:-
Fetch the code.
-
Create a package skeleton.
-
Open the file up in your editor of choice
-
-
For example, to create a new package for
mpileaks
, you type:spack create https://github.com/LLNL/mpileaks/releases/download/v1.0/mpileaks-1.0.tar.gz
-
Spack will create the package file in
$SPACK_ROOT/var/spack/repos/myproject
and name itpackage.py
. Also, it opens the file usingvim
and the file looks like the following:# Copyright 2013-2020 Lawrence Livermore National Security, LLC and other # Spack Project Developers. See the top-level COPYRIGHT file for details. # # SPDX-License-Identifier: (Apache-2.0 OR MIT) # ---------------------------------------------------------------------------- # If you submit this package back to Spack as a pull request, # please first remove this boilerplate and all FIXME comments. # # This is a template package file for Spack. We've put "FIXME" # next to all the things you'll want to change. Once you've handled # them, you can save this file and test your package like this: # # spack install mpileaks # # You can edit this file again by typing: # # spack edit mpileaks # # See the Spack documentation for more information on packaging. # ---------------------------------------------------------------------------- from spack import * class Mpileaks(AutotoolsPackage): """FIXME: Put a proper description of your package here.""" # FIXME: Add a proper url for your package's homepage here. homepage = "https://www.example.com" url = "https://github.com/LLNL/mpileaks/releases/download/v1.0/mpileaks-1.0.tar.gz" # FIXME: Add a list of GitHub accounts to # notify when the package is updated. # maintainers = ['github_user1', 'github_user2'] version('1.0', sha256='2e34cc4505556d1c1f085758e26f2f8eea0972db9382f051b2dcfb1d7d9e1825') # FIXME: Add dependencies if required. # depends_on('foo') def configure_args(self): # FIXME: Add arguments other than --prefix # FIXME: If not needed delete this function args = [] return args
-
Close the file and try to install it using the
spack install
command:spack install mpileaks
-
You will get the following error:
>> 35 configure: error: unable to locate adept-utils installation
-
This error indicates that configure is unable to find the installation location of a dependency.
-
You need to edit the file and add:
-
Some descriptions about the software.
-
Required dependencies.
-
The configuration arguments needed to build the package.
-
Variants.
-
-
If you want to edit the
package.py
file, you can use thespack edit
command:spack edit mpileaks
Adding Package Documentation
-
To add some documentations, you make the following changes:
-
Remove the instructions between dashed lines at the top
-
You add a proper description of the package. Replace the first
FIXME
comment withTool to detect and report MPI objects like MPI_Requests and MPI_Datatypes.
-
Update the
homepage
property with the correct link of the home page of the package. For this example, usehttps://github.com/LLNL/mpileaks
-
Uncomment the maintainers property
and add your GitHub user name.
-
-
Once those changes are done, the file looks like:
from spack import * class Mpileaks(AutotoolsPackage): """Tool to detect and report MPI objects like MPI_Requests and MPI_Datatypes.""" homepage = "https://github.com/LLNL/mpileaks" url = "https://github.com/LLNL/mpileaks/releases/download/v1.0/mpileaks-1.0.tar.gz" maintainers = ['altahat2003'] version('1.0', sha256='2e34cc4505556d1c1f085758e26f2f8eea0972db9382f051b2dcfb1d7d9e1825') # FIXME: Add dependencies if required. # depends_on('foo') def configure_args(self): # FIXME: Add arguments other than --prefix # FIXME: If not needed delete this function args = [] return args
-
The changes won’t help you to build the software. If you try to install it again, you will have the same error you had before.
Adding Dependencies
-
You need to find the dependencies of the software and add them in the form
depends_on('foo')
. After reviewing the documentation ofmpileaks
, you will find that thempileaks
software depends onmpi
,adept-utils
, andcallpath
. You add those dependencies to thepackage.py
file using thedepends_on
directive as follows:depends_on('mpi') depends_on('adept-utils') depends_on('callpath')
-
The file will look like:
from spack import * class Mpileaks(AutotoolsPackage): """Tool to detect and report MPI objects like MPI_Requests and MPI_Datatypes.""" homepage = "https://github.com/LLNL/mpileaks" url = "https://github.com/LLNL/mpileaks/releases/download/v1.0/mpileaks-1.0.tar.gz" maintainers = ['altahat2003'] version('1.0', sha256='2e34cc4505556d1c1f085758e26f2f8eea0972db9382f051b2dcfb1d7d9e1825') depends_on('mpi') depends_on('adept-utils') depends_on('callpath') def configure_args(self): # FIXME: Add arguments other than --prefix # FIXME: If not needed delete this function args = [] return args
|
Dependency Type
-
There are four types of dependencies:
-
build: the dependency will be added to the
PATH
andPYTHONPATH
atbuild-time
. -
link: the dependency will be added to Spack’s compiler wrappers, automatically injecting the appropriate linker flags, including
-I, -L, and RPATH/RUNPATH
handling. -
run: the dependency will be added to the
PATH
andPYTHONPATH
atrun-time
. This is true for bothspack load
and the module files Spack writes. -
test: the dependency will be added to the
PATH
andPYTHONPATH
atbuild-time
. The only difference between build and test is that test dependencies are only built if the user requests unit tests withspack install --test
.
-
-
You can specify dependency type in the
depends_on
directive. For example, the following code makescmake
as build and run dependencies:depends_on('cmake', type='build') depends_on('py-numpy', type=('build', 'run'))
-
If the dependency type isn’t specified, Spack uses a default of ('build', 'link'). This is the common case for compiler languages. Non-compiled packages like Python modules commonly use ('build', 'run'). This means that the compiler wrappers don’t need to inject the dependency’s prefix/lib directory. But the package needs to be in
PATH
andPYTHONPATH
during the build process and later when a user wants to run the package.
Conditional Dependencies
-
Sometimes you may have a package that only requires a dependency under certain conditions. For example, you may have a package that has optional
MPI
support, -MPI
is a dependency only when you want to enableMPI
support for the package. In that case, you could say something like:variant('mpi', default=False, description='Enable MPI support') depends_on('mpi', when='+mpi')
-
You added a variant named
mpi
, and if you enable this variant in the configuration, the package needsmpi
as a dependency.If a dependency/feature of a package isn’t typically used, you can save time by making it conditional (since Spack won’t build the dependency unless it’s required for the Spec).
Specifying Configure Arguments
-
Since the package hasn’t been built successfully, you need to solve the problem. You explore the configure file which is placed in the source directory of the package.
-
To move to the build directory you can use the
spack cd
command:spack cd mpileaks
-
Given this is a simple package built with configure and you know that the installation directories need to be specified.
-
You can use its help to see what command-line options are available for the software.
-
To get details about
configure
file, type:./configure --help
-
The following output shows that you can specify the paths for the two concrete dependencies with the following options:
-
--with-adept-utils=PATH
-
--with-callpath=PATH
-
Now you go back to the
$SPACK_ROOT
and edit thepackage.py
usingspack edit mpileaks
. You add the following lines to theargs
:'--with-adept-utils={0}'.format(self.spec['adept-utils'].prefix), '--with-callpath={0}'.format(self.spec['callpath'].prefix)
-
Now the
package.py
file looks like:from spack import * class Mpileaks(AutotoolsPackage): """Tool to detect and report MPI objects like MPI_Requests and MPI_Datatypes.""" homepage = "https://github.com/LLNL/mpileaks" url = "https://github.com/LLNL/mpileaks/releases/download/v1.0/mpileaks-1.0.tar.gz" maintainers = ['altahat2003'] version('1.0', sha256='2e34cc4505556d1c1f085758e26f2f8eea0972db9382f051b2dcfb1d7d9e1825') depends_on('mpi') depends_on('adept-utils') depends_on('callpath') def configure_args(self): args = [ '--with-adept-utils={0}'.format(self.spec['adept-utils'].prefix), '--with-callpath={0}'.format(self.spec['callpath'].prefix) ] return args
-
Now, if you try to install the package, it will install successfully.
Adding Variants
-
Also, you can add variants to the
package.py
file. Variants represent some arguments of the package. To find what arguments you can add as variants, you look at the help of the configure file as follows:./configure --help
-
The following output shows that there are some optional packages such as
--with-stack-start-c
and--with-gnu-ld
. You will add those options as variants to thepackage.py
file ofmpileaks
. You edit the file again by typing:spack edit mpileaks
-
Then you add the variants as follows:
variant('startframes', values=int, default=0, description='Specify the number of stack frames to truncate') variant('gnuld', default=False, description='assume the C compiler uses GNU ld') depends_on('mpi') depends_on('adept-utils') depends_on('callpath') def configure_args(self): # FIXME: Add arguments other than --prefix # FIXME: If not needed delete this function args = [ '--with-adept-utils={0}'.format(self.spec['adept-utils'].prefix), '--with-callpath={0}'.format(self.spec['callpath'].prefix) ] startframes = int(self.spec.variants['startframes'].value) if startframes: args.extend([ '--with-stack-start-c={0}'.format(startframes) ]) gnuld = int(self.spec.variants['gnuld'].value) if gnuld: args.extend([ '--with-gnu-ld={0}'.format(start), ]) return args
-
First, you add the variants one by one. You choose a proper name for the variant. Also, specify the values of the variant and what’s the default value. You can also give a description of the variant. For example, for
--with-gnu-ld
, you added a variant and named itgnuld
and the default value isFalse
.values
wasn’t specified because it takes only two values;True or False
. But for--with-stack-start-c
, thevalues
was specified asint
because it takes integer values. -
In the
configure_args
function, it checks if the value of the variant isn’t 0, it will be added to the package configuration. If you show the information of thempileaks
package, you see that the added variants are available:spack info mpileaks
Output:
Variants: Name [Default] Allowed values Description =============== ==================== =============================== gnuld [off] on, off assume the C compiler uses GNU ld startframes [0] int(x=0) -> int or Specify the number of stack long int(x, base=10) frames to truncate
-
Variants have been updated in the release 'v0.17.0'. Now you can make the variant conditional by adding a
when=<spec>
clause to the variant. This will allow variants to be conditional based on the version or other attributes of a package. The syntax to model variants that are new in new versions of a package looks like:variant('version_based', default=False, when='@2.0:', description="Variant that is only available in versions 2.0 and later")
-
The directive above means that the variant
version_based
for the package starting at version 2.0. Any command that refers to it for a version that doesn’t satisfy the constraint should fail with an error. -
For example, the following class has a variant
bar
when thefoo
package is at version 2.0 or higher.class Foo(Package): ... variant('bar', default=False, when='@2.0:', description='help message')
-
The syntax to model variants that are dependent on other variants looks like:
variant('variant_based', default=False, when='+version_based', description="Variant that depends on another variant")
-
For example, the package
foo
has a variantbar
when the spec satisfiesplatform=darwin
condition but not other platforms.variant('bar', default=True, when='platform=darwin', description='help2')