|
66
|
1 # Building `sys/unix`
|
|
|
2
|
|
|
3 The sys/unix package provides access to the raw system call interface of the
|
|
|
4 underlying operating system. See: https://godoc.org/golang.org/x/sys/unix
|
|
|
5
|
|
|
6 Porting Go to a new architecture/OS combination or adding syscalls, types, or
|
|
|
7 constants to an existing architecture/OS pair requires some manual effort;
|
|
|
8 however, there are tools that automate much of the process.
|
|
|
9
|
|
|
10 ## Build Systems
|
|
|
11
|
|
|
12 There are currently two ways we generate the necessary files. We are currently
|
|
|
13 migrating the build system to use containers so the builds are reproducible.
|
|
|
14 This is being done on an OS-by-OS basis. Please update this documentation as
|
|
|
15 components of the build system change.
|
|
|
16
|
|
|
17 ### Old Build System (currently for `GOOS != "linux"`)
|
|
|
18
|
|
|
19 The old build system generates the Go files based on the C header files
|
|
|
20 present on your system. This means that files
|
|
|
21 for a given GOOS/GOARCH pair must be generated on a system with that OS and
|
|
|
22 architecture. This also means that the generated code can differ from system
|
|
|
23 to system, based on differences in the header files.
|
|
|
24
|
|
|
25 To avoid this, if you are using the old build system, only generate the Go
|
|
|
26 files on an installation with unmodified header files. It is also important to
|
|
|
27 keep track of which version of the OS the files were generated from (ex.
|
|
|
28 Darwin 14 vs Darwin 15). This makes it easier to track the progress of changes
|
|
|
29 and have each OS upgrade correspond to a single change.
|
|
|
30
|
|
|
31 To build the files for your current OS and architecture, make sure GOOS and
|
|
|
32 GOARCH are set correctly and run `mkall.sh`. This will generate the files for
|
|
|
33 your specific system. Running `mkall.sh -n` shows the commands that will be run.
|
|
|
34
|
|
|
35 Requirements: bash, go
|
|
|
36
|
|
|
37 ### New Build System (currently for `GOOS == "linux"`)
|
|
|
38
|
|
|
39 The new build system uses a Docker container to generate the go files directly
|
|
|
40 from source checkouts of the kernel and various system libraries. This means
|
|
|
41 that on any platform that supports Docker, all the files using the new build
|
|
|
42 system can be generated at once, and generated files will not change based on
|
|
|
43 what the person running the scripts has installed on their computer.
|
|
|
44
|
|
|
45 The OS specific files for the new build system are located in the `${GOOS}`
|
|
|
46 directory, and the build is coordinated by the `${GOOS}/mkall.go` program. When
|
|
|
47 the kernel or system library updates, modify the Dockerfile at
|
|
|
48 `${GOOS}/Dockerfile` to checkout the new release of the source.
|
|
|
49
|
|
|
50 To build all the files under the new build system, you must be on an amd64/Linux
|
|
|
51 system and have your GOOS and GOARCH set accordingly. Running `mkall.sh` will
|
|
|
52 then generate all of the files for all of the GOOS/GOARCH pairs in the new build
|
|
|
53 system. Running `mkall.sh -n` shows the commands that will be run.
|
|
|
54
|
|
|
55 Requirements: bash, go, docker
|
|
|
56
|
|
|
57 ## Component files
|
|
|
58
|
|
|
59 This section describes the various files used in the code generation process.
|
|
|
60 It also contains instructions on how to modify these files to add a new
|
|
|
61 architecture/OS or to add additional syscalls, types, or constants. Note that
|
|
|
62 if you are using the new build system, the scripts/programs cannot be called normally.
|
|
|
63 They must be called from within the docker container.
|
|
|
64
|
|
|
65 ### asm files
|
|
|
66
|
|
|
67 The hand-written assembly file at `asm_${GOOS}_${GOARCH}.s` implements system
|
|
|
68 call dispatch. There are three entry points:
|
|
|
69 ```
|
|
|
70 func Syscall(trap, a1, a2, a3 uintptr) (r1, r2, err uintptr)
|
|
|
71 func Syscall6(trap, a1, a2, a3, a4, a5, a6 uintptr) (r1, r2, err uintptr)
|
|
|
72 func RawSyscall(trap, a1, a2, a3 uintptr) (r1, r2, err uintptr)
|
|
|
73 ```
|
|
|
74 The first and second are the standard ones; they differ only in how many
|
|
|
75 arguments can be passed to the kernel. The third is for low-level use by the
|
|
|
76 ForkExec wrapper. Unlike the first two, it does not call into the scheduler to
|
|
|
77 let it know that a system call is running.
|
|
|
78
|
|
|
79 When porting Go to a new architecture/OS, this file must be implemented for
|
|
|
80 each GOOS/GOARCH pair.
|
|
|
81
|
|
|
82 ### mksysnum
|
|
|
83
|
|
|
84 Mksysnum is a Go program located at `${GOOS}/mksysnum.go` (or `mksysnum_${GOOS}.go`
|
|
|
85 for the old system). This program takes in a list of header files containing the
|
|
|
86 syscall number declarations and parses them to produce the corresponding list of
|
|
|
87 Go numeric constants. See `zsysnum_${GOOS}_${GOARCH}.go` for the generated
|
|
|
88 constants.
|
|
|
89
|
|
|
90 Adding new syscall numbers is mostly done by running the build on a sufficiently
|
|
|
91 new installation of the target OS (or updating the source checkouts for the
|
|
|
92 new build system). However, depending on the OS, you may need to update the
|
|
|
93 parsing in mksysnum.
|
|
|
94
|
|
|
95 ### mksyscall.go
|
|
|
96
|
|
|
97 The `syscall.go`, `syscall_${GOOS}.go`, `syscall_${GOOS}_${GOARCH}.go` are
|
|
|
98 hand-written Go files which implement system calls (for unix, the specific OS,
|
|
|
99 or the specific OS/Architecture pair respectively) that need special handling
|
|
|
100 and list `//sys` comments giving prototypes for ones that can be generated.
|
|
|
101
|
|
|
102 The mksyscall.go program takes the `//sys` and `//sysnb` comments and converts
|
|
|
103 them into syscalls. This requires the name of the prototype in the comment to
|
|
|
104 match a syscall number in the `zsysnum_${GOOS}_${GOARCH}.go` file. The function
|
|
|
105 prototype can be exported (capitalized) or not.
|
|
|
106
|
|
|
107 Adding a new syscall often just requires adding a new `//sys` function prototype
|
|
|
108 with the desired arguments and a capitalized name so it is exported. However, if
|
|
|
109 you want the interface to the syscall to be different, often one will make an
|
|
|
110 unexported `//sys` prototype, and then write a custom wrapper in
|
|
|
111 `syscall_${GOOS}.go`.
|
|
|
112
|
|
|
113 ### types files
|
|
|
114
|
|
|
115 For each OS, there is a hand-written Go file at `${GOOS}/types.go` (or
|
|
|
116 `types_${GOOS}.go` on the old system). This file includes standard C headers and
|
|
|
117 creates Go type aliases to the corresponding C types. The file is then fed
|
|
|
118 through godef to get the Go compatible definitions. Finally, the generated code
|
|
|
119 is fed though mkpost.go to format the code correctly and remove any hidden or
|
|
|
120 private identifiers. This cleaned-up code is written to
|
|
|
121 `ztypes_${GOOS}_${GOARCH}.go`.
|
|
|
122
|
|
|
123 The hardest part about preparing this file is figuring out which headers to
|
|
|
124 include and which symbols need to be `#define`d to get the actual data
|
|
|
125 structures that pass through to the kernel system calls. Some C libraries
|
|
|
126 preset alternate versions for binary compatibility and translate them on the
|
|
|
127 way in and out of system calls, but there is almost always a `#define` that can
|
|
|
128 get the real ones.
|
|
|
129 See `types_darwin.go` and `linux/types.go` for examples.
|
|
|
130
|
|
|
131 To add a new type, add in the necessary include statement at the top of the
|
|
|
132 file (if it is not already there) and add in a type alias line. Note that if
|
|
|
133 your type is significantly different on different architectures, you may need
|
|
|
134 some `#if/#elif` macros in your include statements.
|
|
|
135
|
|
|
136 ### mkerrors.sh
|
|
|
137
|
|
|
138 This script is used to generate the system's various constants. This doesn't
|
|
|
139 just include the error numbers and error strings, but also the signal numbers
|
|
|
140 and a wide variety of miscellaneous constants. The constants come from the list
|
|
|
141 of include files in the `includes_${uname}` variable. A regex then picks out
|
|
|
142 the desired `#define` statements, and generates the corresponding Go constants.
|
|
|
143 The error numbers and strings are generated from `#include <errno.h>`, and the
|
|
|
144 signal numbers and strings are generated from `#include <signal.h>`. All of
|
|
|
145 these constants are written to `zerrors_${GOOS}_${GOARCH}.go` via a C program,
|
|
|
146 `_errors.c`, which prints out all the constants.
|
|
|
147
|
|
|
148 To add a constant, add the header that includes it to the appropriate variable.
|
|
|
149 Then, edit the regex (if necessary) to match the desired constant. Avoid making
|
|
|
150 the regex too broad to avoid matching unintended constants.
|
|
|
151
|
|
|
152 ### internal/mkmerge
|
|
|
153
|
|
|
154 This program is used to extract duplicate const, func, and type declarations
|
|
|
155 from the generated architecture-specific files listed below, and merge these
|
|
|
156 into a common file for each OS.
|
|
|
157
|
|
|
158 The merge is performed in the following steps:
|
|
|
159 1. Construct the set of common code that is idential in all architecture-specific files.
|
|
|
160 2. Write this common code to the merged file.
|
|
|
161 3. Remove the common code from all architecture-specific files.
|
|
|
162
|
|
|
163
|
|
|
164 ## Generated files
|
|
|
165
|
|
|
166 ### `zerrors_${GOOS}_${GOARCH}.go`
|
|
|
167
|
|
|
168 A file containing all of the system's generated error numbers, error strings,
|
|
|
169 signal numbers, and constants. Generated by `mkerrors.sh` (see above).
|
|
|
170
|
|
|
171 ### `zsyscall_${GOOS}_${GOARCH}.go`
|
|
|
172
|
|
|
173 A file containing all the generated syscalls for a specific GOOS and GOARCH.
|
|
|
174 Generated by `mksyscall.go` (see above).
|
|
|
175
|
|
|
176 ### `zsysnum_${GOOS}_${GOARCH}.go`
|
|
|
177
|
|
|
178 A list of numeric constants for all the syscall number of the specific GOOS
|
|
|
179 and GOARCH. Generated by mksysnum (see above).
|
|
|
180
|
|
|
181 ### `ztypes_${GOOS}_${GOARCH}.go`
|
|
|
182
|
|
|
183 A file containing Go types for passing into (or returning from) syscalls.
|
|
|
184 Generated by godefs and the types file (see above).
|
