KNOWLEDGE BASE ON CAMLCITY.ORG: omake
How to use the OMake build utility best - by Gerd Stolpmann, 2012-11-19
Starting point: A simple OMakefile for a program could look like:
USE_OCAMLFIND = true OCAMLPACKS = lwt FILES = foo bar .DEFAULT: $(OCamlProgram myprog, $(FILES))For a library, just use
.DEFAULT: $(OCamlLibrary mylib, $(FILES))instead of the
OCamlProgramline. The following recipes are, in some sense, variations of these simple patterns.
Normally, you just only set variables to control the build. Some of the variables only affect the linker, some only the compiler, and a number both (here the important ones):
|USE_OCAMLFIND||X||X||true/false - whether to use findlib|
|OCAMLINCLUDES||X||X||search path for compiled modules and archives, also used by the dependency scanner|
|OCAMLFINDFLAGS||X||X||additional flags to pass to ocamlfind|
|OCAMLPACKS||X||X||findlib packages to include in build|
|BYTE_ENABLED||X||X||true/false - whether bytecode compiler is enabled|
|NATIVE_ENABLED||X||X||true/false - whether nativecode compiler is enabled|
|OCAMLCFLAGS||X||X||flags to pass to "ocamlc"|
|OCAMLOPTFLAGS||X||X||flags to pass to "ocamlopt"|
|OCAMLFLAGS||X||X||flags to pass to "ocamlc" and "ocamlopt"|
|OCAML_BYTE_LINK_FLAGS||X||flags to pass to "ocamlc" when linking an executable|
|OCAML_NATIVE_LINK_FLAGS||X||flags to pass to "ocamlopt" when linking an executable|
|OCAML_LINK_FLAGS||X||flags to pass to both "ocamlc" and "ocamlopt" when linking executables|
|OCAML_LIBS||X||library archives of the project to link into executables|
|OCAML_OTHER_LIBS||X||library archives outside the project to link into executables|
|OCAML_LIB_FLAGS||X||extra flags when linking libraries|
For example, this OMakefile builds two programs from a different set of files, and links different libraries into the programs:
USE_OCAMLFIND = true OCAMLPACKS = lwt section FILES = foo bar .DEFAULT: $(OCamlProgram myprog1, $(FILES)) section FILES = bar baz .DEFAULT: $(OCamlProgram myprog2, $(FILES))Basically, it makes sense to set linking-related variables that are interpreted by
OCAML_LIBS(see "Starting point" for a list of variables interpreted at link time).
What does not work here is to set variables that (only or also) affect
the compiler (i.e. "ocamlc -c" and "ocamlopt -c"). If you e.g. set
OCAMLPACKS within a
section, there will be no effect on compilation (only on
linking). Below you find another recipe that shows how to do that, but
this simple recipe here does not allow it.
root +--- dir1 +--- dir2and you want to build in all three directories. How can we describe this?
Solution 1: Multiple OMakefiles
Here, only root contains OMakeroot, and all three directories have
their own OMakefile. Also, you need to "chain" the files: In
root/OMakefile, there needs to be a
# Before .SUBDIRS: Put here definitions that are visible in the whole project: USE_OCAMLFIND = true OCAMLPACKS = lwt .SUBDIRS: dir1 dir2 # After .SUBDIRS: Add here definitions that are only applied in the root dir: FILES = mainprog OCAMLINCLUDES = dir1 dir2 OCAML_LIBS = dir1/lib1 dir2/lib2 .DEFAULT: $(OCamlProgram myprog, $(FILES))Now, put into dir1/OMakefile and dir2/OMakefile the definitions that are local to these directories, e.g.
FILES = file1 files OCAMLFLAGS = -g OCAMLPACKS += netstring .DEFAULT: $(OCamlLibrary lib1, $(FILES))Basically, the definitions in the sub-OMakefiles have their own scoping environment, as if they were encapsulated in a
section. The scoping rules for directories and for sections are not identical, though. In particular, you can also define in a sub-OMakefile variables that affect the compilation, like
Solution 2: .SUBDIRS with body
Sometimes, you don't want to put OMakefiles into subdirectories, or the OMakefiles in several directories would be identical. To handle this case, it is allowed to put the definitions for the subdirectory into the body of .SUBDIRS (i.e. indented):
# Before .SUBDIRS: Put here definitions that are visible in the whole project: USE_OCAMLFIND = true OCAMLPACKS = lwt .SUBDIRS: dir1 dir2 .DEFAULT: $(OCamlProgram myprog, main)This works exactly as if you had written the definitions in the body into separate OMakefiles in the subdirs.
OCAMLFINDFLAGS += -syntax camlp4o OCAMLPACKS += camlp4(or use "-syntax camlp4r" if you prefer the revised syntax). In order to activate a custom preprocessor written for camlp4, like xstrp4, just add this findlib package to
OCAMLPACKS += xstrp4Note that the variable
OCAMLFINDFLAGSnot only affects ocamlc and ocamlopt, but also ocamldep, i.e. the dependency scanning is also done with camlp4 and the activated syntax extensions.
Some custom preprocessors accept command-line arguments. For example, you can define macros on the command-line with the macro preprocessor:
OCAMLPACKS += camlp4.macro # the macro preprocessor OCAMLFINDFLAGS += $(mapprefix -ppopt, -D NAME=value)The strange "mapprefix" expression just prepends
-ppoptto every space-separated part of the second argument - well, in the end, camlp4 is called with
-D NAME=valueappended to the command-line.
If you don't trust in ocamlfind, you can also directly set the preprocessor command, e.g.
OCAMLFLAGS += -pp camlp4ofbut custom extensions like
xstrp4can then not be enabled by simply adding the package to
OCAMLPACKS. Instead, you would have to find out the right options for camlp4of, and add them to the command passed with -pp.
As camlp4 reduces the compilation speed tremendously, you often want to enable it only for a few files, but not for the whole project. See the following recipe how to do this.
sectionconstruct normally does not allow to modify compiler flags, but only linker flags. You may for example want to do this to invoke a preprocessor only for certain source files.
Note that you do not need this technique if you just set the compiler flags in a sub-OMakefile to different values than in the main OMakefile, because OMake already handles this case automatically.
The compiler flags have an effect on the standard build rules, which are defined by pattern rules like
%.cmo: %.ml $(OCamlC) -c $<(the actual standard definitions are a lot more complicated), where
OCamlCis a variable that also expands the compiler flags like
OCAMLINCLUDES. Normally, the build rules just take the values these flags have at the end of the OMakefile. If you now want to force OMake to take the values set in a certain
section, the trick is to recall the build rule within the section. Here is an example:
section OCAMLINCLUDES = additional_include_dir foo.cmi: foo.cmo: foo.cmx: foo.o:This means that the module
foois compiled with the changed compiler flags. Note that we omit the bodies of the build rules - which OMake interprets to leave the body unchanged (i.e. take it from the pattern), and only to apply the current set of variables to the old body.
One application is to instruct OMake to compile certain modules with a preprocessor. The following defines a function doing this:
Camlp4o(module) = section OCAMLPACKS += camlp4 OCAMLFINDFLAGS += -syntax camlp4o $(module).cmi: $(module).cmo: $(module).o: $(module).cmx:You can call this function for every module as in
Camlp4o(mod1) Camlp4o(mod2)to enable camlp4 just for these modules.
Note that you need to name the modules explicitly that get a deviating set of compiler flags. You can, however, use an iteration to apply the deviating set to all modules you deal with in a section:
section FILES = foo bar baz CAMLP4_FILES = bar baz OCAML_LIBS = mylib1 mylib2 foreach(f, $(CAMLP4_FILES)) Camlp4o($(f)) .DEFAULT: $(OCamlProgram myprog1, $(FILES))
Not limited to camlp4: This is, of course, not limited to requiring camlp4. Another example it to increase the inlining value just for a certain file:
Inline(module,limit) = section OCAMLOPTFLAGS += -inline $(limit) $(module).cmi: $(module).cmo: $(module).o: $(module).cmx: Inline(mod1, 10) Inline(mod2, 15)
root +--- directory1 # containing lib1 +--- directory2 # containing lib2It is very simple to just use a library while building another (or building a program). E.g. if you need lib1 to build lib2, just set in
OCAMLINCLUDES = ../directory1to make the library available.
OCAMLINCLUDESis a far-reaching variable: the directories enumerated in it are not only added as
-Iflags in the compiler commands to run, but also considered for dependency scanning. That means when a source file in directory1 is changed, the modules that depend on it in directory2 are automatically found out, and added to the list of things to recompile.
If you build a program using the libraries, you have to do a tiny
additional step: In the OMakefile building the program also set
OCAML_LIBS in addition to
OCAML_LIBS needs to be set to the paths to the library
archives (i.e. to the cma/cmxa files, but omit the suffix). For example,
if there are programs to build in
OCAMLINCLUDES = directory1 directory2 OCAML_LIBS = directory1/archive1 directory2/archive2to link these archives while building the programs.
OMake will automatically take care of the dependencies between the
libraries, and even between individual files. However, there is
a disadvantage of doing only this: OMake does not provide a means
of defining indirect dependencies well. For example, if you have
lib2, and the latter uses
lib3, you need to list
lib3 as a prerequisite
lib1, even if there is no direct relationship.
For large projects with dozens of libraries this turns out to be a
big problem, because such "expanded" dependencies are unmaintainable.
Solution 1: Define the dependencies only in the main OMakefile.
The idea is that there is a paragraph at the beginning of the main OMakefile describing the dependencies, e.g. if we have a directory hierarchy
root +--- directory1 # contains lib1 +--- directory2 # contains lib2 +--- directory3 # contains lib3 +--- directory4 # build something elsewe can define a number of variables in
LIB3_INCLUDES = $(dir directory3) LIB2_INCLUDES = $(dir directory2) $(LIB3_INCLUDES) LIB1_INCLUDES = $(dir directory1) $(LIB2_INCLUDES)and in the sub-OMakefile for a certain library we set only
OCAMLINCLUDESto one of the prepared variables, e.g. if lib4 needs lib1, we set in
OCAMLINCLUDES = $(LIB1_INCLUDES)
Why did we call the function
dir while setting
LIB*_INCLUDES variables? Remember that we use the
LIB1_INCLUDES in a subdirectory, so
directory2 is here
../directory2. If we invoke
the variable magically rembers where it was defined, and if it is
recalled from a different directory, OMake automatically adjusts
(Detail: You might have noticed
LIB1_INCLUDES is actually an array of an array. This does not
harm us here, because when OMake expands such structured variables to
command-line arguments, it just flattens them to a list of arguments.)
For building programs, we also need the library archives. We can use the same pattern as for the include path:
LIB3_ARCHIVES = $(file directory3/archive3) LIB2_ARCHIVES = $(file directory2/archive2) $(LIB3_ARCHIVES) LIB1_ARCHIVES = $(file directory1/archive1) $(LIB2_ARCHIVES)(Note that we need here to call
For creating an executable linking in lib1 and all predecessors, just use
OCAMLINCLUDES = $(LIB1_INCLUDES) OCAML_LIBS = $(LIB1_ARCHIVES)If the library archives have all the same name, you can also do:
OCAMLINCLUDES = $(LIB1_INCLUDES) OCAML_LIBS = $(addsuffix /archivename, $(LIB1_INCLUDES))If the archives have the same name as the directories:
mapname(name) = value $(name)/$(basename $(name)) OCAMLINCLUDES = $(LIB1_INCLUDES) OCAML_LIBS = $(foreach $(mapname), $(LIB1_INCLUDES))
Solution 2 (advanced): Extract the dependency data from META files.
Here we assume that we have already META files in the subdirectories. We want to use ocamlfind to look up paths and archive names:
root +--- directory1 # contains lib1 | +---- META +--- directory2 # contains lib2 | +---- META +--- directory3 # contains lib3 | +---- META +--- directory4 # contains lib4 +---- METAThe first thing to do is to set
OCAMLPATHso it contains the
setenv(OCAMLPATH, $(absname .):$(getenv OCAMLPATH,$(string)))
absname returns the absolute path name
of the argument.
$(string) is just the empty string. In OMake
there is no good way of writing the empty string as a quoted literal.)
Now we can refer to the libraries as packages. The package names are
just the directory names, e.g. "directory3" would mean lib3. It is,
however, not sufficient to just add these names to
because OMake would then not consider these libraries as part of the
project, but as packages outside the project.
The solution is to still set
OCAMLINCLUDES in the
sub-OMakefiles like in solution 1. This time, however, we call
ocamlfind for getting the values. To do so, define
root/OMakefile before the
all_dirs = $(subdirs .) get_project_findlib_dirs(pkg) = dirs = $(shella ocamlfind query -r $(pkg)) fdirs = foreach (d, $(dirs)) if $(mem $(dir $(d)), $(all_dirs)) fdirs += $(d) export export value $(fdirs) get_project_findlib_archives(pkg,pred) = dirs = $(shella ocamlfind query -format %d/%a -predicates $(pred) -r $(pkg)) fdirs = foreach (d, $(dirs)) if $(mem $(dir $(d)), $(all_dirs)) fdirs += $(d) export export value $(fdirs)Note that these functions only output directories and archives inside the project.
(Detail: Note that the
mem function is used to
check whether a directory
d occurs in a list of
all_dirs. This works independently of the
path by which the directories are referenced. Actually,
is normally a relative path, and
absolute paths. Cool, isn't it.)
How to use this: In the sub-OMakefiles that create libraries:
OCAMLPACKS += directory1 directory2 OCAMLINCLUDES = $(get_project_findlib_dirs $(OCAMLPACKS))
OCAMLPACKS line the project-local findlib
packages are added to the build. The
is only necessary to instruct the scanner that the dependencies
to these packages are also needed.
In the sub-OMakefiles that create programs (here
OCAMLPACKS += directory1 directory2 program.run: $(get_project_findlib_archives $(OCAMLPACKS), byte) program.opt: $(get_project_findlib_archives $(OCAMLPACKS), native)You need the two "program.run"/"program.opt" lines for every program you create. The effect is that the library archives of the project-local packages are added as dependency to the build.
public.InterfaceDoc(name, files) = protected.mlifiles = $(filter-exists $(addsuffix .mli, $(files))) protected.cmifiles = $(addsuffix .cmi, $(removesuffix $(mlifiles))) $(name).idoc: $(mlifiles) $(cmifiles) /.PHONY/OCamlGeneratedFilesTarget ocamlfind ocamldoc -dump $(name).idoc -stars \ $(PREFIXED_OCAMLINCLUDES) -package "$(OCAMLPACKS)" \ $(mlifiles) return $(name).idocYou call this function like this in every OMakefile compiling
InterfaceDoc(filename, $(FILES))This creates a file
filename.idoc(idoc for interface documentation) in the current directory. This file is a binary description of the documentation, and it can be used in a final ocamldoc step (modify this as needed):
IDOCS = subdirectory1/filename1.idoc subdirectory2/filename2.idoc .PHONY: html-doc html-doc: $(IDOCS) mkdir -p html rm -rf html/* ocamlfind ocamldoc -d html -stars -t "My title" -html \ $(mapprefix -load, $(IDOCS))
$ omake BYTE_ENABLED=truethis setting is often ignored. The reason for this is that omake interprets command-line assignments just as initial assignments, but not as overriding assignments like traditional "make". If your OMakefile contains already something like
BYTE_ENABLED = $(not $(OCAMLOPT_EXISTS))this will simply overwrite the initial setting given on the command-line.
Note that the standard build definitions, as in
OCaml.om (part of the OMake deployment) also
set many control variables. These settings are, however, overridable.
Look at OMakeroot to see why:
open build/C open build/OCaml open build/LaTeX DefineCommandVars() .SUBDIRS: .In English, this just means that the three standard build definition files are loaded, and first after that the command-line assignments are evaluated. Finally, the
.SUBDIRS: .directive includes the OMakefile in the same directory.
So, what to do? If you have a block at the beginning of your OMakefile where you customize variables, like
BYTE_ENABLED = true NATIVE_ENABLED = $(OCAMLOPT_EXISTS) OCAMLPACKS = p1 p2 p3you just call
DefineCommandVarsagain after this block:
DefineCommandVars()This evaluates the assignments given on the command-line again, and the caller can override your customizations.
There is also the possibility to set variables only if they aren't already set. Unfortunately, OMake does not know the popular "make" syntax "variable ?= value", and we have to do it on our own:
if $(not $(defined X)) X = default_value exportThere are other formulations:
var_default(name, value) = if $(not $(defined $(name))) setvar($(name), $(value)) export export var_default(X, default_value)Another option, closer to a normal assignment:
var_default(value, name) = # different order! if $(not $(defined $(name))) setvar($(name), $(value)) export export var_default(X) default_value
root +---- directory1_src +---- directory1_out +---- directory2_src +---- directory2_outand we want that the output files of the sources in
*_out. This is achieved by changing the
vmount(-l, directory1_src, directory1_out) # "minus ell" as first arg vmount(-l, directory2_src, directory2_out) .SUBDIRS: directory1_out directory2_outThe effect of
vmountis that the files in the source directories are automatically linked into the output directories. After that, you do not refer to the source directories anymore, but just to the output directories.
Building several versions of the same: You can use the vmount feature to build several output directories from the same source directory, and have different configurations for each output. Example:
section vmount(-l, directory1_src, directory1_normal) ENABLE_DEBUG = false .SUBDIRS: directory1_normal section vmount(-l, directory1_src, directory1_debug) ENABLE_DEBUG = true .SUBDIRS: directory1_debugOf course, both versions share the same OMakefile (residing in
directory1_src). Because of this, we set one variable
ENABLE_DEBUGdifferently in both versions. You can now use conditionals like
if $(ENABLE_DEBUG) OCAMLFINDFLAGS += $(mapprefix -ppopt, -D DEBUG) # set a camlp4 macro for IFDEFin
directory1_src/OMakefileto enable the debug code if the OMakefile is interpreted in the "right" output directory.
Note that the OMake developers consider vmount still as experimental.
if $(defined ENABLE_PROF) OCAMLOPTFLAGS += -p exportIf the user calls omake on the command-line as
omake ENABLE_PROF=anyvaluethe feature is enabled.
Simultaneous profile and non-profile builds: If you develop a library for the general public, you may want to provide both a non-profile and a profile build. Given the following directory layout
root +---- directory1_src +---- directory1_normal +---- directory1_gprofyou can use the vmount feature to enable the profile build only in directory1_gprof. (We assume here that you do not build anything in
root/OMakefilereplace the normal
section vmount(-l, directory1_src, directory1_normal) ENABLE_PROF = false .SUBDIRS: directory1_normal section vmount(-l, directory1_src, directory1_gprof) ENABLE_PROF = true .SUBDIRS: directory1_gprofPut all your sources into
directory1_src, and leave the other two directories empty. The
if $(defined ENABLE_PROF) OCAMLOPTFLAGS += -p BYTE_ENABLED = false exportat the beginning. If you now build your projects, you get everything twice: In
directory1_normalprofiling is disabled, whereas in
directory1_gprofit is enabled.
When installing your project as findlib library, there is the
possibility to put all archive and cmi files into the same findlib
directory. The cmi files should be the same in both versions, so you
can take them from either version. The native archives, however, are
different (both the .cmxa and the .a files). An install target
root/OMakefile) could e.g. do
.PHONY: install install: ln-or-cp directory1_gprof/myarchive.cmxa directory1_gprof/myarchive.p.cmxa ln-or-cp directory1_gprof/myarchive.a directory1_gprof/myarchive.p.a ocamlfind install mylib META \ directory1_src/*.mli \ directory1_normal/*.cmi \ directory1_normal/myarchive.cma \ directory1_normal/myarchive.cmxa \ directory1_normal/myarchive.a \ directory1_gprof/myarchive.p.cmxa \ directory1_gprof/myarchive.p.aand the META file would refer to the archives like
archive(byte) = "myarchive.cma" archive(native) = "myarchive.cmxa" archive(native,gprof) = "myarchive.p.cmxa"This way, the profile-enabled version of your library is automatically taken when
ocamlfind ocamlopt -pruns.
Note that the OMake developers consider vmount still as experimental.
MENHIR_ENABLED=true, files ending in .mly are instead processed by menhir
atdgen utility can generate I/O serializers
for given type definitions. If called like
atdgen -t foo.atdit generates from the description
foo.atdtwo files, namely
foo_t.mli. We create this rule for it:
%_t.ml %_t.mli: %.atd atdgen -t $<Note that you should describe all output files on the left side of the colon, and all input files on the right side. In the command,
$<is replaced by the leftmost input file.
It most cases, adding such a rule is already fully sufficient to
drive the generation, and to include the generated files into the
dependency graph. OMake calls ocamldep with the
flag meaning that it emits the dependencies by module names and not by
files. Because of that, ocamldep also prints dependencies on modules
that do not yet exist as files, but first need to be generated. OMake
then postprocesses these dependencies, and compares them with the
left sides of the rules to check whether the files can be generated.
The problem here is that OMake has its own idea what the C compiler to call is, and with which flags. With a few configurations, though, one can tell OMake to call the right C compiler with the right flags:
CC_CONFIG = $(shell $(OCAMLC) -config | grep bytecode_c_compiler:) CC = $(nth 1, $(CC_CONFIG)) CFLAGS = $(nth-tl 2, $(CC_CONFIG)) CFLAGS += -I$(shell $(OCAMLC) -where)These are essentially the same flags that are added when you compile a C file and use ocamlc as driver (e.g.
ocamlc -c file.c).
In a native-only project you can replace "bytecode_c_compiler" with "native_c_compiler" (this avoids the -fPIC flag giving a minimal boost).
The built-in link function
OCamlLibrary does not support
stub libraries. Because of this we need to define our own rules. Given
the following source files
foo.ml foo.mli foo_stubs.cwe would like to generate these outputs:
libfoo.a # containing foo_stubs.o libfoo.so foo.cma # the compiled foo.ml foo.cmxaThe rules doing this:
OCAML_MKLIB_FLAGS = ... # set e.g. to: -lbar if $(BYTE_ENABLED) foo.cma libfoo$(EXT_LIB): foo.cmo foo_stubs$(EXT_OBJ) ocamlmklib -o foo foo.cmo foo_stubs$(EXT_OBJ) \ $(OCAML_BYTE_LINK_FLAGS) $(OCAML_LINK_FLAGS) $(OCAML_MKLIB_FLAGS) if $(NATIVE_ENABLED) foo.cmxa libfoo$(EXT_LIB): foo.cmx foo_stubs$(EXT_OBJ) ocamlmklib -custom -o foo foo.cmx foo_stubs$(EXT_OBJ) \ $(OCAML_NATIVE_LINK_FLAGS) $(OCAML_LINK_FLAGS) $(OCAML_MKLIB_FLAGS)Note that we do not specify that libfoo.so (or libfoo.dll on Windows) is also created, at least on many platforms.
echo A : Bdoes not output "A : B", but rather defines a dependency that "echo" and "A" depend on "B". How to circumvent:
X = A : B).
Sometimes, however, I compile a project with a different ocaml version than the last time. Now I get errors, even for "omake clean", e.g.:
*** omake: reading OMakefiles *** omake error: File /opt/godi-4.00/lib/omake/build/C.om: line 4, characters 2-32 unbound variable: public.OMakeVersionThe problem is that OMake caches some values between invocations, namely in the files ending in
.omakedbin the topmost directory
.DEFAULT: $(OCamlProgram ...)if the program is to built by default, but only
OCamlProgram(...)if not. Essentially, OMake distinguishes between contexts where you invoke functions as expressions returning values, and contexts where functions are only seen as statements, returning nothing.
On the outermost level, everything is a statement, e.g.
VAR = expr
file.html: file.txt transform-txt-to-html file.txt
if $(condition) statement1 else statement2
$(if $(condition),$(getenv V1),$(getenv V2))
value $(expression...)(in function definitions you can also use
returnif you want to leave the function immediately). And, of course, expressions may have side effects, e.g.
$(setvar VAR, $(...))sets
VARin the same way as the statement
VAR=$(...)would have. In some sense, the duality between "statement syntax" and "expression syntax" exists to keep some compatibility with traditional "make".
Now back to our initial question.
OCamlProgram is a
function that defines a number of build rules, and finally returns the
main targets as an array of strings. If you
OCamlProgram as a statement, the returned array
is silently dropped. If you invoke it as an expression, as in
.DEFAULT: $(OCamlProgram prog, $(FILES))the function returns the array [| "prog"; "prog.opt"; "prog.run" |] after definining the build rules as side effect, and this is the same as if you had written
OCamlProgram(prog, $(FILES)) .DEFAULT: prog prog.opt prog.runeffectively declaring these targets as default targets.