The Craftr build system¶

Craftr is a next generation build system based on Ninja and Python that features modular and cross-platform build definitions at the flexibility of a Python script and provides access to multiple levels of build automation abstraction.

Features
Cross-platform	Use Craftr anywhere Ninja and Python can run
Modular build scripts	Import other build scripts as Python modules, synergizes well with `git submodule`
Performance	Craftr outperforms traditional tools like Make, CMake or Meson
Language-independent	Don't be tied to a single language, use Craftr for anything you want!
Extensive standard library	High-level interfaces to modern C/C++ compilers, C+, Java, Protobuff, Yacc, Cython, Flex, NVCC (OpenCL coming soon)
Everything under your control	Use the lowlevel API when- and wherever you need it and manully define the exact build commands
Consequent out-of-tree builds	Craftr never builds in your working tree (unless you tell it to)

Requirements¶

Ninja
Python 3.4 or newer

Contents¶

Getting Started¶

Craftr is built around Python-ish modules that we call Craftr modules or Craftfiles (though this name usually refers to the first type of Craftr modules). There are two ways a Craftr module can be created:

A file named Craftfile.py with a # craftr_module(...) declaration
A file named craftr.ext.<module_name>.py where <module_name> is of course the name of your Craftr module

By default, Craftr will execute the Craftfile.py from the current working directy if no different main module is specified with the -m option. Below you can find a simple Craftfile that can build a C++ program on any platform (that is supported by the Craftr STL).

# craftr_module(my_project)
from craftr import path
from craftr.ext import platform

# Create object files for each .cpp file in the src/ directory.
obj = platform.cxx.compile(
  sources = path.glob('src/*.cpp'),
  std = 'c++11',
)

# Link all object files into an executable called "main".
program = platform.ld.link(
  inputs = obj,
  output = 'main'
)

Below is a sample invokation on Windows. We pass the -v flag for additional debug output by Craftr and full command-line output from Ninja.

λ craftr -v
detected ninja v1.6.0
cd "build"
load 'craftr.ext.my_project'
(craftr.ext.my_project, line 9): unused options for compile(): {'std'}
exporting 'build.ninja'
$ ninja -v
[1/2] cl /nologo /c c:\users\niklas\desktop\test\src\main.cpp /Foc:\users\niklas\desktop\test\build\my_project\obj\main.obj /EHsc /W4 /Od /showIncludes
[2/2] link /nologo c:\users\niklas\desktop\test\build\my_project\obj\main.obj /OUT:c:\users\niklas\desktop\test\build\my_project\main.exe

λ ls build build\my_project\
build:
build.ninja  my_project/

build\my_project\:
main.exe*  obj/

Installation¶

pip install craftr-build

To install from the Git repository, use the -e flag to be able to update Craftr by simply pulling the latest changes from the remote repository.

git clone https://github.com/craftr-build/craftr.git && cd craftr
pip install -e .

Targets¶

Craftr describes builds with the craftr.Target class. Similar to rules in Makefiles, a target has input and output files and a command to produce the output files. Note that in Craftr, targets can also represents Tasks which can be used to embed real Python functions into the build graph.

Using the Target class directly is usually not necessary unless you have very specific requirements and need control over the exact commands that will be executed. Or if you’re just being super lazy and need the easiest script to compile a C program:

# craftr_module(super_lazy)
from craftr import Target, path
main = Target(
  command = 'gcc $in -o $out',
  inputs  = path.local(['src/main.c', 'src/util.c']),
  outputs = 'main'
)

The substition of $in and $out is conveniently done by Ninja.

$ craftr .main
[1/1] gcc /home/niklas/Desktop/example/src/main....til.c -o /home/niklas/Desktop/example/build/main

Tasks¶

Tasks were initially designed as functions doing convenient operations that can be invoked from the command-line, they can also be used to export any function as a “command” to the Ninja manifest and have the production of output files implemented in Python.

A common use-case for tasks is to generate an archive from the build products to have it ready for distribution. Below you can find a simple example using the archive and git extension modules:

#craftr_module(myapp)
from craftr import path, task, info
from craftr.ext import archive, git, platform

git = git.Git(project_dir)
obj = platform.cc.compile(sources = path.glob('src/*.c'))
bin = platform.ld.linkn(inputs = obj, output = 'myapp')

@task(requires = [bin])
def archive():
  archive = Archive(prefix = 'myapp-{}'.format(git.describe()))
  archive.add('res')        # Add a directory to the archive
  archive.add(bin.outputs)  # Add the produced binary
  archive.save()
  info('archive saved: {!r}'.format(archive.name))

Note

Craftr is clever enough to run a task directly if it doesn’t need any Ninja targets to be built before it can be executed. For example, the following task via craftr .hello

@task
def hello():
  info('Hello, World!')

Generator Functions¶

Most of the time you don’t want to be using Targets directly but instead use functions to produce them with a high-level interface. It is sometimes useful to create such a target generator function first and then use it to reduce the complexity of the build script.

The Craftr standard library provides an extensive set of functions and classes that generate targets for you, most notably the C/C++ compiler toolsets.

Frameworks¶

The craftr.Framework is in fact just a dictionary (with an additional name attribute) that represents a set of options for anything build related. How the data is interpreted depends on the generator function.

Frameworks are useful for grouping build information. They were designed for C/C++ builds but may find other uses as well. For example, there might be a framework for a C++ library that specifies the include paths, preprocessor definitions, linker search path and other libraries required for the library to be used in a C++ application.

For example, the Craftfile for a header-only C++ library might look as simple as this:

from craftr import Framework, path
from craftr.ext.libs.some_library import some_library
my_library = Framework(
  frameworks = [some_library],
  include = [path.local('include')],
  libs = ['zip'],
)

As you can see in the example above, frameworks can also be nested.

Targets there were generated by helper functions (as described in the Generator Functions section) list up the frameworks that have been used to produce the target in the Target.frameworks attribute. Passing a target directly as input to another generator function will automatically inherit the frameworks of that target!

fw = Framework(
  include = [path.local('vendor/include'),
  libpath = [path.local('vendor/lib')],
  libs = ['vendorlib1', 'vendorlib2']
)

obj = cc.compiler(
  sources = path.glob('src/*.c'),
  frameworks = [fw]
)

bin = ld.link(
  inputs = obj
  # we don't need to specify "fw" again, it is inherited from "obj"
)

Build Options¶

Options for the build process are entirely read from environment variables. The craftr.options.get() function is a convenient method to read the options from the environment.

In Craftr, options can be specified local for a module or globally for all modules. A local option is actually prefixed by the full name of the Craftr module.

#craftr_module(app)
from craftr import options
name = options.get('name')
debug = options.get_bool('debug')

info('Hello {}, you want a {} build?'.format(name, 'debug' if debug else 'release'))

The options can be specified locally using the following methods:

craftr -D.name="John Doe" -D.debug
craftr -Dapp.name="John Doe" -Dapp.debug
app.name="John Doe" app.debug="true" craftr   # assuming your shell supports this syntax

They can be set globally like this:

craftr -Dname="John Doe" -Ddebug
name="John Doe" debug="true" craftr   # assuming your shell supports this syntax

Options and environment variables can also be set from craftrc.py files.

Oh, and say hello to John!

Hello John Doe, you want a debug build?

craftrc.py Files¶

Before anything, Craftr will execute a craftrc.py file if any exist. This file can exist in the current working directory and/or the users home directory. Both will be executed if both exist! You can prevent Craftr from executing these files by passing --no-rc. You can also tell it to execute a specific file with the --rc parameter (can be combined).

This file is not executed in a Craftr module context and hence should not declare any targets, but it can be used to set up environment variables and options.

For example, for using the craftr.ext.qt5 module on Windows, you could use this craftrc.py file in the home directory to let the Craftr Qt5 module know where the Qt5 headers and libraries are located.

from os import environ
if 'Qt5Path' not in environ:
  environ['Qt5Path'] = 'D:\\lib\\Qt\\5.5\\msvc2013_64'

Note that you can still specify a different Qt5Path via the command line that will override the value set in the craftrc.py file because the environment variables are set in the following order:

Variables from the parent process/shell
Variables prefixed on the command-line (like VAR=VALUE craftr ...) if your shell supports it
craftrc.py files that modify the craftr.environ
Options passed via the -D, --define command-line parameter
Craftr modules that modify the craftr.environ

Colorized Output¶

Craftr colorizes output by default if it is attached to a TTY. If it is not but colorized output is still desired, CRAFTR_ISATTY can be set to true in the environment. Also, colorized output can be disabled by setting the variable to false instead. For any other value, default behaviour applies.

Debugging¶

Not only can you debug your Craftr build scripts with the pdb module, but you can also increase the verbosity level for more verbose output. This is very useful for tracing down warnings or locations of errors in the output, eg.:

λ craftr --skip-build
you really shouldn't do it that way!

To find the location of that line, we can pass -v.

λ craftr --skip-build -v
detected ninja v1.6.0
cd "build"
load 'craftr.ext.test'
(craftr.ext.test, line 4): you really shouldn't do it that way!
exporting 'build.ninja'

Now if you’re really having trouble finding out how the Python script actually gets there, you can enable a stacktrace with each line that is output with -vv.

λ craftr --skip-build -vv
detected ninja v1.6.0
cd "build"
load 'craftr.ext.test'
(craftr.ext.test, line 4): you really shouldn't do it that way!
  In <module> (F:\Python34\Scripts\craftr-script.py, line 9)
  In main() (c:\users\niklas\repos\craftr-build\craftr\craftr\__main__.py, line 256)
  In import_module() (f:\python34\lib\importlib\__init__.py, line 109)
  In load_module() (c:\users\niklas\repos\craftr-build\craftr\craftr\ext.py, line 245)
  In <craftr.ext.test> (Craftfile.py, line 4)
exporting 'build.ninja'

This output is also nicely colorized if you’re in a terminal that supports ANSI color codes.