Scons Tool Util¶

Welcome to scons-tool-util documentation!

Purpose¶

SCons Tool Util package is a library of reusable functions and objects for SCons tools’ developers.

Table of Contents¶

User documentation¶

Introduction¶

The SCons Tool Utilities package is a library of reusable functions and objects designed to be used by newly developed SCons tools.

Utilities¶

ToolFinder¶

Description¶

ToolFinder is a configurable functor used to search for executable files within predefined search paths. Once created (configured), an instance of ToolFinder is expected to return same search result, every time it’s applied to a given SCons environment.

python = ToolFinder('python')
def generate(env):
   env['PYTHON'] = python(env)   # same result here ...
def exists(env):
   return python(env)            # ... and here

The program being searched for is identified by a name. By default, tool name is used ('python' in the above example). This may be overwritten with the name parameter

python = ToolFinder('python', name='python3')

The above finder will search for 'python3' file for the purpose of SCons tool named 'python'. Cumbersome strings may be used for name, including variable substitutions, like name="$PYTHONNAME".

python = ToolFinder('python', name='$PYTHONNAME')

A sequence of file names may also be passed in as name (alternative names for the program).

# Will search for 'python3', then 'python', then 'python2' in each path
python = ToolFinder('python', name=['python3', 'python', 'python2'])

By default, ToolFinder searches within the standard SCons PATH (env['ENV']['PATH']). This can be changed, by providing path argument to the constructor

# Will search in custom path, instead of env['ENV']['PATH']
python = ToolFinder('python', path=['/home/user/.local/virtualenvs/foo/bin',
                                    '/home/user/.local/bin'])
prog = python(env)

If file is found in path (or SCons PATH, if path not given), its name is returned. By setting strip_path to False, the object is being told to return absolute path instead

# Will return absolute path to the file found
python = ToolFinder('python', strip_path=False)
prog = python(env)    # '/usr/bin/python', for example

ToolFinder accepts two extra search paths: priority_path, and fallback_path. The priority_path is searched prior to the SCons PATH, the fallback_path is examined after the SCons PATH.

# Will search in priority_path, then env['ENV']['PATH'], then fallback_path.
python = ToolFinder('python', priority_path=['/home/user/.local/bin'],
                              fallback_path=['/opt/bin'])
prog = python(env)

The priority-/fallback- paths are used by ToolFinder at the tool configuration time, but it’s assumed that they’re not used by SCons during the build phase (later), when the program has to be invoked. Following this assumption, to ensure that same program will be picked up at the build stage, ToolFinder returns an absolute path to any program found within either priority_path or fallback_path. This behavior may be changed with strip_priority_path and strip_fallback_path, respectively

# Will return 'python', instead of '/home/user/.local/bin/python'
python = ToolFinder('python', priority_path=['/home/user/.local/bin'],
                              strip_priority_path=True)
prog = python(env)

Examples¶

Iconv tool¶

The iconv(1) command translates texts between encodings. The following simple tool makes use of iconv(1) installed in standard location (within SCons search PATH).

Example:	Tool implementation

Tool module: site_scons/site_tools/iconv.py¶

# -*- coding: utf-8 -*-
from sconstool.util import *
from SCons.Builder import Builder

_iconv = ToolFinder('iconv')


def generate(env):
    env.SetDefault(ICONVFROM='UTF-8', ICONVTO='UTF-8')
    env.SetDefault(ICONV=_iconv(env))
    env['ICONVCOM'] = '$ICONV -f $ICONVFROM -t $ICONVTO $SOURCE > $TARGET'
    env['BUILDERS']['Iconv'] = Builder(action='$ICONVCOM')


def exists(env):
    return _iconv(env)

Example:	A project using the `iconv` tool

Project file: SConstruct¶

# SConstruct
import os
env = Environment(tools=['iconv'], ENV={'PATH': os.environ['PATH']})
env.Iconv('utf8.txt', 'latin2.txt', ICONVFROM='LATIN2')

Input file: latin2.txt¶

Zaźółć tę gęślą jaźń

Hammer tool¶

This tool will use our own python script hammer.py stored in a non-standard path. The hammer.py will replace all the occurrences of "nail" with "drived in nail".

Example:	The `hammer` command

Command implmementation: bin/hammer.py¶

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import sys
with open(sys.argv[2], 'rt') as ifile, open(sys.argv[1], 'wt') as ofile:
    ofile.write(ifile.read().replace('nail', 'drived in nail'))

Example:	Tool implementation

The tool module: site_scons/site_tool/hammer.py¶

# -*- coding: utf-8 -*-
from sconstool.util import *
from SCons.Builder import Builder
import sys

_hammer = ToolFinder('hammer', name='hammer.py', pathext=['.py'], priority_path='bin')


def generate(env):
    env.SetDefault(PYTHON=sys.executable)
    env.SetDefault(HAMMER=_hammer(env))
    env['HAMMERCOM'] = '$PYTHON $HAMMER $TARGET $SOURCE'
    env['BUILDERS']['Hammer'] = Builder(action='$HAMMERCOM')


def exists(env):
    return _hammer(env)

Example:	A project using the `hammer` tool.

Project file: SConstruct¶

# SConstruct
env = Environment(tools=['hammer'])
env.Hammer('output.txt', 'input.txt')

Input file: input.txt¶

We have twenty nails.

ConditionalEmitter¶

Description¶

ConditionalEmitter is a callable object which dispatches control between two user-provided functions: emitter_if and emitter_else, both having the following prototype:

emitter(target, source, env)

The ConditionalEmitter object is designed to be used as a SCons emitter. The decision, which of the user-provided functions shall be called, is made based on a user-defined condition, which, again, is provided by user in the form of a predicate function

predicate(target, source, env)

The predicate shall return a boolean value.

In short, the following conditional emitter

em = ConditionalEmitter(predicate, emitter_if, emitter_else)

when used as

res = em(target, source, env)

will return the result of emitter_if(target, source, env) if predicate(target, source, env) is True, or will return the result of emitter_else(target, source, env) otherwise.

Selector¶

Description¶

Selector can be used to replace SCons.Util.Selector which currently seems to have some quirks. While SCons.Util.Selector works only with simple suffixes (such as .yyy), without embedded dots, the Selector handles long, multi-part suffixes (such as .xxx.yyy).

ReplacingBuilder¶

Description¶

ReplacingBuilder is a wrapper for SCons builders, used to call the wrapped builders with certain construction variables replaced. A mapping of original variable names into their replacements is provided via constructor arguments.

obj = ReplacingBuilder(env['BUILDERS']['Object'], CFLAGS='MY_CFLAGS')
env['BUILDERS']['MyObject'] = obj

In the above example, a builder named MyObject is created which passes the value of MY_CFLAGS instead of CFLAGS to the Object builder. When used as follows with gcc

env.Replace(MY_CFLAGS=['-Wall', '-Wextra'])
env.MyObject('test.c')

it will invoke compiler with -Wall and -Wextra flags

gcc -c -o test.o -Wall -Wextra test.c

Replacements are applied also to variables passed via builder’s keyword arguments

env.MyObject('test.c', MY_CFLAGS=['-Wall', '-Wextra'])

The ReplacingBuilder wrapper exposes inject_replacements() method which may be used to set the replacement variables in environment to their default values.

obj = ReplacingBuilder(env['BUILDERS']['Object'], CFLAGS='MY_CFLAGS')
env['BUILDERS']['MyObject'] = obj
obj.inject_replacements(env)
assert env['MY_CFLAGS'] == '$CFLAGS'

Note, that replacements also alter source and target prefixes/suffixes. Let’s redefine MyObject builder as follows

obj = ReplacingBuilder(env['BUILDERS']['Object'], OBJSUFFIX='MY_OBJSUFFIX')
env['BUILDERS']['MyObject'] = obj
env['MY_OBJSUFFIX'] = '.my$OBJSUFFIX'

this builder will produce files with suffix '.my.o' if the original $OBJSUFFIX is '.o'. Note, that when you wrap your own builders, they should use original variables, like $OBJSUFFIX for suffixes, not their replacements.

obj = SCons.Builder.Builder(action=SCons.Defaults.CXXAction,
                            emitter={},
                            prefix='$OBJPREFIX',   # <- not $MY_OBJPREFIX
                            suffix='$OBJSUFFIX',   # <- not $MY_OBJSUFFIX
                            src_builder=['MyCXXFile'],
                            src_suffix='$MY_CXXSUFFIX',
                            source_scanner=SCons.Tool.SourceFileScanner,
                            single_source=1)
obj = ReplacingBuilder(obj, OBJPREFIX='MY_OBJPREFIX',
                            OBJSUFFIX='MY_OBJSUFFIX')
env['BUILDERS']['MyObject'] = obj
# ...
env.SetDefault(MY_OBJSUFFIX='.my$OBJSUFFIX')

If we used suffix='$MY_OBJSUFFIX' in the above example, variable substitution would be performed twice, and the actual suffix woule be '.my.my.o' instead of '.my.o'.

Examples¶

Shared library builder for SWIG-generated python modules¶

The following example is a modified extract from scons-tool-swigpy tool. The presented code implements a shared library builder named SwigPyShlib which generates shared library (or dll) with prefix '_' (SWIG convention for generated Python modules) and suffix '.pyd' (Windows convention for Python extension modules). The infixes for SwigPyShlib, as well as few other values will be provided via SWIGPY_* variables. Instead of using SWIG to generate hello_wrap.c file, we write such a file by hand. Instead of loading _hello.pyd as a python module, we’ll write simple test.c program that will load _hello.pyd at runtime.

Example:	Tool implementation

Tool module: site_scons/site_tools/swigpy.py¶

# -*- coding: utf-8 -*-
from sconstool.util import *
import SCons.Tool
import SCons.Builder

SwigPyVars = [
  'LINK',
  'LINKFLAGS',
  'LIBPATH',
  'LIBS',
  'SHOBJPREFIX',
  'SHOBJSUFFIX',
  'LIBPREFIX',
  'LIBSUFFIX',
  'SHLIBPREFIX',
  'SHLIBSUFFIX',
  'IMPLIBPREFIX',
  'IMPLIBSUFFIX',
  'WINDOWSEXPPREFIX',
  'WINDOWSEXPSUFFIX'
]


SwigPyReplacements = Replacements({k: 'SWIGPY_%s' % k for k in SwigPyVars })


class SwigPyShlibBuilder(ReplacingBuilder):
    def __call__(self, env, target, source, **kw):
        # preserve original 'LIBSUFFIXES' and 'LIBPREFIXES', such that
        # libraries having original 'LIBPREFIX', 'LIBSUFFIX', 'SHLIBPREFIX',
        # etc. will be found when required by the linker.
        ovr = {'LIBPREFIXES': [env.subst(x) for x in env['LIBPREFIXES']],
               'LIBSUFFIXES': [env.subst(x) for x in env['LIBSUFFIXES']]}
        return ReplacingBuilder.__call__(self, env, target, source, **dict(ovr, **kw))


def createSwigPyShlibBuilder(env):
    try:
        swigpy_shlib = env['BUILDERS']['SwigPyShlib']
    except KeyError:
        shlib = SCons.Tool.createSharedLibBuilder(env)
        swigpy_shlib = SwigPyShlibBuilder(shlib, SwigPyReplacements)
        env['BUILDERS']['SwigPyShlib'] = swigpy_shlib
    return swigpy_shlib


def setSwigPyDefaults(env):
    env.SetDefault(SWIGPY_SHLIBPREFIX='_')
    env.SetDefault(SWIGPY_LIBPREFIX='_')
    env.SetDefault(SWIGPY_IMPLIBPREFIX='_')
    env.SetDefault(SWIGPY_WINDOWSEXPPREFIX='_')
    env.SetDefault(SWIGPY_SHLIBSUFFIX='.pyd')
    SwigPyReplacements.inject(env, 'SetDefault')


def generate(env):
    createSwigPyShlibBuilder(env)
    setSwigPyDefaults(env)


def exists(env):
    return 1

Example:	A project using the `swigpy` tool

Project file: SConstruct¶

# SConstruct
import sys
env = Environment(tools=['default', 'swigpy'])
env.Append(SWIGPY_LIBPATH=['.'])
env.SharedLibrary('hello', 'hello.c', CPPDEFINES={'BUILDING_HELLO': 1})
env.SwigPyShlib('hello', 'hello_wrap.c', SWIGPY_LIBS=['hello'], CPPDEFINES={'BUILDING_HELLO_WRAP': 1})
if sys.platform == 'win32':
  libs = []
else:
  libs = ['dl']

env.Program('test_hello.c', LIBS=libs)

C file: hello.c¶

#include <stdio.h>
#include "hello.h"
void HELLO_API hello() {
  printf("hello" EOL); fflush(stdout);
}

Header file: hello.h¶

#ifndef HELLO_H
#define HELLO_H

#ifdef _WIN32
# ifdef BUILDING_HELLO
#  define HELLO_API __declspec(dllexport)
# else
#  define HELLO_API __declspec(dllimport)
# endif
# define EOL "\r\n"
#else
# define HELLO_API
# define EOL "\n"
#endif

#ifdef __cplusplus
extern "C" {
#endif

extern void HELLO_API hello();

#ifdef __cplusplus
}
#endif
#endif

Wrapper C file: hello_wrap.c¶

#include "hello.h"
#include "hello_wrap.h"
#include <stdio.h>
void HELLO_WRAP_API hello_wrap() {
  printf("wrap" EOL "  ");  fflush(stdout);
  hello();
  printf("unwrap" EOL);     fflush(stdout);
}

Header file: hello_wrap.h¶

#ifndef HELLO_WRAP_H
#define HELLO_WRAP_H

#ifdef _WIN32
# ifdef BUILDING_HELLO_WRAP
#  define HELLO_WRAP_API __declspec(dllexport)
# else
#  define HELLO_WRAP_API __declspec(dllimport)
# endif
#else
# define HELLO_WRAP_API
#endif

#ifdef __cplusplus
extern "C" {
#endif

extern void HELLO_WRAP_API hello_wrap();

#ifdef __cplusplus
}
#endif

#endif /* HELLO_WRAP_H */

Test program: test_hello.c¶

#ifdef _WIN32
# include <windows.h>
typedef HINSTANCE lib_t;
# define OpenLib(_s) LoadLibrary(_s)
# define OpenLib_str "LoadLibrary"
# define LoadLibSym(_lib,_s) GetProcAddress(_lib, _s)
# define LoadLibSym_str "GetProcAddress"
#else
# include <dlfcn.h>
typedef void* lib_t;
# define OpenLib(_s) dlopen(_s, RTLD_NOW)
# define OpenLib_str "dlopen"
# define LoadLibSym(_lib,_s) dlsym(_lib,_s)
# define LoadLibSym_str "dlsym"
#endif

#include <stddef.h>
#include <stdio.h>
#include <stdlib.h>

#ifdef _WIN32

void error_exit(LPTSTR func)
{
    // Retrieve the system error message for the last-error code

    LPVOID msg;

    FormatMessageA( FORMAT_MESSAGE_ALLOCATE_BUFFER |
                    FORMAT_MESSAGE_FROM_SYSTEM |
                    FORMAT_MESSAGE_IGNORE_INSERTS,
                    NULL,
                    GetLastError(),
                    MAKELANGID(LANG_ENGLISH, SUBLANG_ENGLISH_US),
                    (LPTSTR) &msg,
                    0, NULL );
    fprintf(stderr, "%s() failed: %s\r\n", func, (LPTSTR)msg);
    LocalFree(msg);

    exit(EXIT_FAILURE);
}

#else

void error_exit(char const* func)
{
    fprintf(stderr, "%s() failed: %s\n", func, dlerror());
    exit(EXIT_FAILURE);
}

#endif

typedef void(*void_fcn_t)();

void_fcn_t load_hello_wrap()
{
  void_fcn_t hello_wrap;
  lib_t pyd = OpenLib("_hello.pyd");
  if(!pyd) {
    error_exit(OpenLib_str);
  }
  hello_wrap = (void_fcn_t)LoadLibSym(pyd, "hello_wrap");
  if(!hello_wrap) {
    error_exit(LoadLibSym_str);
  }
  return hello_wrap;
}

int main(int argc, const char* argv[])
{
  void_fcn_t hello_wrap = load_hello_wrap();
  hello_wrap();
  return 0;
}

Testing on Linux¶

$ scons
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
gcc -o hello_wrap.os -c -fPIC -DBUILDING_HELLO_WRAP=1 hello_wrap.c
gcc -o hello.os -c -fPIC -DBUILDING_HELLO=1 hello.c
gcc -o libhello.so -shared hello.os
gcc -o _hello.pyd -shared hello_wrap.os -L. -lhello
gcc -o test_hello.o -c test_hello.c
gcc -o test_hello test_hello.o -ldl
scons: done building targets.
$ LD_LIBRARY_PATH='.' ./test_hello
wrap
  hello
unwrap

API Documentation¶

This is an official API documentation for the scons-tool-util python package.

Introduction¶

For specific documentation, please skip to the next sections.

The sconstool.util is an implicit namespaced package (see PEP 420) within sconstool namespace. Because of this, there is no lib/sconstool/__init__.py.

Classes¶

This section documents classes provided by scons-tool-util package. The summary below provides links to a full documentation for each class.

`ToolFinder`(tool, **kw)	Callable object which searches for executables.
`ConditionalEmitter`(predicate[, emitter_if, …])	A callable object, which calls user-provided emitter when a predefined condition is meet.
`Selector`	A replacement for `SCons.Util.Selector` class.
`Replacements`	Enables one to temporarily replace variables in a dict-like object (SCons `Environment`).
`ReplacingCaller`(wrapped[, replacements])	Base class for `ReplacingBuilder`, `ReplacingAction` and other similar wrappers.
`ReplacingBuilder`(wrapped[, replacements])	SCons builder wrapper, calls the wrapped builder with replaced construction variables.
`ReplacingAction`(wrapped[, replacements])	SCons action wrapper, replaces construction variables and calls the wrapped action.

Functions¶

This section documents functions provided by scons-tool-util package. The summary below provides links to a full documentation for each function.

`add_ro_dict_property`(cls, dictattr, attr[, …])	Add to class cls a read-only property returning a predefined entry from a dict.
`ensure_kwarg_in`(caller, key, allowed)	Checks a single key from keyword arguments against allowed keys.
`ensure_kwarg_not_in`(caller, key, forbidden)	Checks a single key from keyword arguments against forbidden keys.
`check_kwarg`(caller, key[, allowed, forbidden])	Checks a single key from keyword arguments against allowed and forbidden keys.
`check_kwargs`(caller, kw[, allowed, forbidden])	Checks all keys from keyword arguments kw against allowed and forbidden keys.
`import_all_from`(target, modules[, …])	Imports symbols from multiple modules.

scons-tool-util - notes for developers¶

This module is designed to be developed with the help of pipenv.

Initialization¶

On a fresh clone do:

pipenv install --dev
pipenv run bin/downloads.py
pipenv run pip install -e .

Running tests¶

There are some end-to-end tests. They can be ran this way:

pipenv run python runtest -e test/system

Unit tests may also be executed, for example:

pipenv run python -m unittest discover -t . -s test/unit

All tests may be executed at once

pipenv run python runtest.py -e -a

Creating package for distribution¶

pipenv run python setup.py sdist bdist_wheel

Uploading to test.pypi.org ¶

pipenv run twine upload -r testpypi dist/*

Uploading to pypi.org ¶

pipenv run twine upload dist/*

Generating HTML documentation¶

pipenv run make html

The generated documentation is writen to build/docs/html, with the index file build/docs/html/index.html.

LICENSE¶

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE

Table of Contents

Related Topics

Scons Tool Util¶

Purpose¶

Table of Contents¶

User documentation¶

Introduction¶

Utilities¶

ToolFinder¶

Description¶

Examples¶

ConditionalEmitter¶

Description¶

Selector¶

Description¶

ReplacingBuilder¶

Description¶

Examples¶

API Documentation¶

Introduction¶

Classes¶

Functions¶

scons-tool-util - notes for developers¶

Initialization¶

Running tests¶

Creating package for distribution¶

Uploading to test.pypi.org ¶

Uploading to pypi.org ¶

Generating HTML documentation¶

LICENSE¶

Indices and tables¶

Scons Tool Util¶

Purpose¶

Table of Contents¶

User documentation¶

Introduction¶

Utilities¶

ToolFinder¶

Description¶

Examples¶

ConditionalEmitter¶

Description¶

Selector¶

Description¶

ReplacingBuilder¶

Description¶

Examples¶

API Documentation¶

Introduction¶

Classes¶

Functions¶

scons-tool-util - notes for developers¶

Initialization¶

Running tests¶

Creating package for distribution¶

Uploading to test.pypi.org¶

Uploading to pypi.org¶

Generating HTML documentation¶

LICENSE¶

Indices and tables¶

Uploading to test.pypi.org ¶

Uploading to pypi.org ¶