Go

Preface Code path: src/runtime/proc.go Data structures are defined under src/runtime/runtime2.go. Concepts Copied from source code: // G - goroutine. // M - worker thread, or machine. // P - processor, a resource that is required to execute Go code. // M must have an associated P to execute Go code, however it can be // blocked or in a syscall w/o an associated P. main Set up stack size. Fork a new m and run the sysmon function, except the wasm platform. Lock the main m and g. Run the runtime initTask’s. Enable GC. Run the main initTask’s. Run main_main. runtime.sysmon sysmon is a global worker runs regularly to ensure the system state consistency. ...

Path: src/runtime/chan.go Compilation When you make a chan with make function, the compiler will expand the expression to the makechan implementation. The actual expansion happens at cmd/compile/internal/walk/expr.go. The runtime will determine whether to use makechan or makechan64. Type Definitions type _type _type is used as the internal representation of a go type. The same structure is defined multiple times across the go runtime. _type stores the following fields: type chantype makechan only uses chantype.elem, the other fields are used by the type system. ...

ssagen.Compile Build SSA for the ir.Func (buildssa) Check ssaDump and set the dump flag. Initialize ssagen.state. Push the line number or the parent’s line number (if it’s missing) to the stack. Set up ssagen.state flags from ir.Func information. Set up an empty ssagen.ssafn with ir.Func and ABI information. Allocate the starting block for the current ssa.Func. Check open-coded defers. <- What’s this? Do ABIAnalyze, get the abi.ABIParamResultInfo of function in/out parameters. Generate the addresses of the local variables saved in ir.Func.Dcl. Generate the AuxCall for the current function. Generate the addresses of the input parameters saved in ir.Func.Dcl. Generate the addresses of the closure variables saved in ir.Func.Dcl. Covert to SSA IR. I don’t have any experience with SSA, so it’s really difficult to understand this part of the code. Insert Phi values. <- What’s this? Call ssa.Compile. It defines an array of ssa.pass, and run each of them against the ssa.Func. Generate SSA (genssa) This step returns a objw.Progs to save the machine-level instructions of the function. objw saves platform-independent structures of the binary. ...

irgen.generate Check each file’s pragma list and DeclList. Generate ir.Decl for type declarations. Generate ir.Decl for other declarations. Process later functions, this step is for the same purpose as type check. Type-check CallExpr again. Check missing function bodies. Build generic instantiations. It scans calls and generated needed methods. Remove all generic Decl’s. After irgen.generate, g.target.Decls will be the final Decl’s to generate. enqueueFunc and compileFunctions This step is the compile step of cmd/compile, it happens after type checking and IR generation. ...

noder.LoadPackage Parsing noder.LoadPackage will call syntax.Parse to generate the syntax tree of all package files. Initialize a compile/internal/syntax.scanner Advance scanner to the next token. Start parsing. Check the package declaration first, and take the package name from the parsed pragma. Note that the parser is expecting a ; token, but this is automatically generated at the scanner, so users don’t have to write the ;. And parsing errors will not always abort the parser loop. Parse top-level declarations: const type var func Each type has its function to parse the whole syntax tree. Type Checking After syntax.Parse, check2 is called to do the IR generation. ...

BuildToolchain BuildToolchain is initialized as noToolchain{} by default, then it’s set dynamically to gccgo or gc. Both implementations are wrappers around the binary toolchains installed on the machine. For gc, the delegations are as follows: BuildToolchain.gc - base.Tool("compile") BuildToolchain.cc - base.Tool("cgo"). Actually cgo has been executed before the compile and cfiles has been cleared, so BuildToolchain.cc should never be called and it always returns an error. BuildToolchain.asm - base.Tool("asm") BuildToolchain.pack - base.Tool("pack") BuildToolchain.ld - base.Tool("link") cmd/compile aka base.Tool(“compile”) Entry point: internal/gc.Main It’s a standalone binary which means all packages path assumes a relative path to go/src/cmd/compile. gc.Main will get an archInit function as the parameter, this function populates the ssagen.ArchInfo which contains necessary architecture information. ...

Builder.build Builder.build generate an archive for a single package. Builder.build gets building information from two sources: internal/cfg package. It contains some global variables saving build flags. load.Package. It contains information for the current building package. Builder.build will build a need flag, finish the step and uncheck the finished bits one by one. Set up caching information by action, its package, and package’s target. Set up objects output directory. Setup cgo cache & vet cache. Set up building target directory. Set up non-Go files overlay. This step is to copy the non-Go files to the object directory. Preprocess coverage files and replace the file path of original *.go files. It runs go tool cover -mode=b.coverMode -var="varName" -o dst.go src.go to generate the coverage files. Build cgo files. After the build, the cfiles will be cleared and generated go files will be added to gofiles Cache all source files. It’s all *.go files now Generate import configuration and package dependencies information. Generate embedded files configuration. Run BuildToolchain.gc. This is the actual go build stage. Run BuildToolchain.cc. This is the actual cgo build stage. Run BuildToolchain.asm. This is the actual *.s build stage. Run BuildToolchain.pack. This will generate the object archive (*.a). Update the BuildID accordingly. Builder.link Builder.link links a package and generates the final binary. It’s very simple compared with the build stage. ...

Set Up the Environment vim-go doesn’t support travesal through the go source code, so I switch to vscode and it works out of box (with Go plugin && gopls installed). Start Point (go run) File: src/cmd/go/main.go Declaration: run.CmdRun Function: run.runRun Key data structures: build.Context work.Builder load.Package work.Action The runRun function will go through several stages: Check shouldUseOutsideModuleMode. This procedure is used by go run cmd@version work.BuildInit. This will setup the build context: Initialize modload module. It will check go module flags and set up the flags, no actual module download. instrumentInit and buildModeInit, and update the default build.Context accordingly. Get a work.Builder. work.NewBuilder will check the environment and make sure it’s ready to do the actual build. Inits a load.Package from all *.go files passed to go run. load.Package has a public struct for definitions, and an internal struct for running state. Setup builder.LinkAction. LinkAction will call cacheAction (for looking up action cache, not build cache), CompileAction, and installAction (not for go run). work.Action is a DAG, the action cache depends on mode and package info. Initialize a work.Action, use the link action initialized at the above stage as dependencies. Build the work.Action with builder. Build Stage (Builder.Do) Set up the cache trim. There’s a trim.txt inside go-build cache folder, it’s used to track the timestamp of last trim action. Build the action list, visit the DAG as “depth-first post-order travelsal”. The priority will set by the list order, which means deepest will run first. writeActionGraph. Internal feature, this will dump the DAG as JSON. Set up triggers. Triggers are the inverse of dependencies, which means when the dependency ready, it will trigger its root instead of its dependencies. The handle function will do the actual jobs. Actions are run in parallel, the actual job is defined by action.Func. After actions done, update the global state. There’s a lock to make sure there’s no data races. If all dependencies finished, push the action node to ready queue and signal the readySema. Ready queue are protected with the same global state lock. Run all actions from the DAG in parallel. What’s Next Travesal through the action.Func definitions: ...