It has been more than 3 years (!) since I wrote the initial blog, so many things have changed now. One of my initial motivations is that “I need a place to hold all those non-that-interesting stuffs”, but now I’ve switch to Zotero, and this becomes a non-issue, Zotero is perfect tool to organize all pieces into a single place. So what now then? I guess I don’t need to drain my energy to read all those boring Go source to code just to keep my “promise” that I started a series, and it’s time to write something really matters to me and that I want to share with others. They don’t have to be “cool”, or “correct”, or “matters to others”, they’re just some pieces of thought. ...

Code Path: src/runtime/cgocall.go The key part here is to switch the stack between Go and CGO Go -> C cgocall entersyscall: this will ensure that CGO codes will not block the Go runtime. mark m as incgo asmcgocall KeepAlive CGO-related fn, arg, and mp asmcgocall Set-up the stack. C -> Go cgocallback Set-up the stack for cgocallbackg, after cgocallbackg, the sp will point to the previous code called cgocallback cgocallbackg callbackUpdateSystemStack lockOSThread to pin the g on the current m exitsyscall cgocallbackg1 undo the previous steps reentersyscall

Code Path: src/runtime/panic.go Ref: Defer, Panic, and Recover Defer Two methods of defer Copied from the comment: The older way involves creating a defer record at the time that a defer statement is executing and adding it to a defer chain. This chain is inspected by the deferreturn call at all function exits in order to run the appropriate defer calls. A cheaper way (which we call open-coded defers) is used for functions in which no defer statements occur in loops. In that case, we simply store the defer function/arg information into specific stack slots at the point of each defer statement, as well as setting a bit in a bitmask. At each function exit, we add inline code to directly make the appropriate defer calls based on the bitmask and fn/arg information stored on the stack. During panic/Goexit processing, the appropriate defer calls are made using extra funcdata info that indicates the exact stack slots that contain the bitmask and defer fn/args. ...

Install LDAP Before install ldap, set-up a valid FQDN for your hostname: Edit /etc/hostname, for example void.kassiansun.com Restart the host If you’ve installed ldap before, purge them all: sudo apt-get remove --purge slapd ldap-utils -y Now we can install the ldap packages: sudo apt-get install slapd ldap-utils -y During the installation, it will prompt to set the default password. Test that you now have a valid LDAP tree: # Output: # dn: # namingContexts: dc=kassiansun,dc=com ldapsearch -H ldap://localhost -x -LLL -s base -b "" namingContexts Clean-Up Old apache2 and php installation sudo apt-get remove --purge apache2 phpldapadmin php* If you’re not using apache or php on your machine, clean them all so we can get started from the scratch. ...

The Generic Set Implementation Code Path: src/t_set.c It’s a wrapper around the actual set type. The set can be one of the following types: An intset. Even if the member is a string, Redis will try to convert it to an integer with string2ll. A listpack. Used for small sets. A dict. Used for large sets. Listpack Implementation The same implementation of hash type. Dict Implementation The same implementation of hash type. ...

The Generic List Implementation Code Path: src/t_list.c It’s a Redis command wrapper of listpack and quicklist. Listpack Implementation The same implementation of hash type. Quicklist Implementation Code Path: src/quicklist.c If the size of the list exceeds a certain number, Redis will try to convert a listpack to quicklist. A quicklist is a double-linked list of listpack: If the data is too large, it will store the data as a “plain” node, without encoding it as listpack. If the data size exceeds the size limit of listpack, it will create a new list node to store it. If it’s in the middle of the node, the node will split into two nodes. If the list node is empty (after deletion), it will get removed from the list. The list node can be compressed, but not the head or the tail. The pop operation is a combination of get & delete.

The Generic Hash Implementation Code Path: src/t_hash.c The generic hash type is a wrapper around listpack and dict, it will switch between two different implementation and call the routines accordingly. Listpack Implementation Code Path: src/listpack.c Listpack is a densed storage of a series of keys and values, it supports storing both integers and strings. If the hash size is lower than hash_max_listpack_value, it will be saved as a listpack. If the hash size is higher than hash_max_listpack_value, it will create a dict and copy the data from listpack to the dict. ...

Code Path: src/runtime/map.go makemap Each hmap has a hash0 as seed hash. When the compiler generates the typing information for a map, the hasher function will be calculated with genhash function (code path: src/cmd/compile/internal/reflectdata/alg.go). If the map is not escaped, the hmap will be allocated by the compiler on the stack. Then makemap will allocate an array of bucket, the type of bucket is generated at compiling stage by MapBucketType (code path: src/cmd/compile/internal/reflectdata/reflect.go). ...

Start-Up Procedures spt_init: This procedure initializes the process’s name and deep copy command-line arguments & environments. tzset. zmalloc_set_oom_handler init_genrand64 with timestamp and pid number. Code Path: mt19937-64.c crc64_init. Code Path: crc64.c umask dictSetHashFunctionSeed with a random 16-byte seed. Code Path: dict.c initServerConfig ACLInit. Code Path: acl.c moduleInitModulesSystem. Code Path: module.c connTypeInitialize. Code Path: connection.c initSentinelConfig and initSentinel if the server was started in sentinel mode. Code Path: sentinel.c Run redis_check_aof_main or redis_check_rdb_main and exit. Code Path: redis-check-aof.c, redis-check-rdb.c Parse command-line options and loadServerConfig. Code Path: config.c. sentinelCheckConfigFile if it’s in sentinel mode. Code Path: sentinel.c Check whether the server is supervised by upstart or systemd. daemonize if required. initServer. It initializes the global redisServer structure. createPidFile if required. redisSetProcTitle if required. checkTcpBacklogSettings. clusterInit if it’s in cluster mode. Code Path: cluster.c moduleLoadFromQueue. Code Path: module.c. ACLLoadUsersAtStartup, Code path: acl.c. initListeners. Redis supports multiple types of listeners, and each one has its own accept handling logic. The listeners will be registered by aeCreateFileEvent clusterInitListeners if it’s in cluster mode. Code Path: cluster.c bioInit initializes the redis’s background I/O. Code Path: bio.c initThreadedIO initializes the threaded I/O. Code Path: networking.c set_jemalloc_bg_thread. If it’s not sentinel mode: aofLoadManifestFromDisk. Code Path: aof.c loadDataFromDisk. It will either load the AOF file with loadAppendOnlyFiles, or load the RDB file with rdbLoad and initialize the replication backlog with createReplicationBacklog. RDB Code Path: rdb.c aofOpenIfNeededOnServerStart to open the AOF file on disk. Code Path: aof.c aofDelHistoryFiles to clear the history_aof_list. Code Path: aof.c verifyClusterConfigWithData check the cluster slots. Note that cluster mode only allows data in db 0. Code Path: cluster.c Else, do sentinelIsRunning. redisSetCpuAffinity setOOMScoreAdj will update the value in /proc/self/oom_score_adj aeMain starts the event loop. aeDeleteEventLoop after the event loop is finished. Code Path: ae.c Event Loop server.el got initialized during initServer. The main event loop is created, serverCron and module_pipe events are registered. beforeSleep and afterSleep are also configured on the event loop. ...