Patching Projects#
This section solves all problems that arise if you want to customize the actual Pod of a project beyond the limits of what license quotas can do. Also, this customization is specific to individual projects, not across all of them!
To make this possible, there are Licenses which contain a JSON Patch. The basic idea is that this license contains a JSON string of a series of such JSON patch operations, which are aggregated across all licenses (if there is more than one) and applied to the project pod right before it is sent to the Kubernetes API.
This is the sequence leading to the final project pod:
During HELM deployment, the
manage/templates/_project.tpl
template is rendered and the resulting JSON is stored in theproject-pod
ConfigMap. If you are audacious, you could deploy your ownproject-pod
ConfigMap with a different template!When a project is issued to start, the “manage” service reads the template from the
project-pod
ConfigMap.It replaces placeholders:
{PROJECT_IMAGE}
the docker image to use for the project{PROJECT_TAG}
the docker image tag to use for the project{project_id}
the project’s UUID, as defined by CoCalc in the database{extra_env}
additional environment variables to set{project_config}
the project configuration, as defined by CoCalc in the database
When the licenses are processed, a quota is computed.
This adjust the CPU and memory requests and limits of the project pod.
Exact details depend on the
max_upgrades
anddefault_quotas
server settings.A license quota could contain a “patch” as well.
The combined list of all patches found in the licenses is applied to the project pod template after all the above happened. CoCalc uses the
applyPatch
function of fast-json-patch.
Example#
As admin, create a “Site License” and in the JSON Patch field, add this:
[
{
"op": "add",
"path": "/metadata/labels/mylabel",
"value": "test123"
}
]
This operation adds a label to the metadata of the project pod. See RFC 6902 for more information.
Then, add this license to a project, which will restart it.
As a result, the project pod is now:
$ kubectl get pod -o yaml project-[UUID]
apiVersion: v1
kind: Pod
metadata:
annotations: {...}
labels:
mylabel: test123 # <<< that's new
network: outside
project_id: [UUID]
project_tag: project-20230110-1143
run: project
[...]
How to create a patch?#
You could the fast-json-patch library to create a patch:
npm init -y
npm i fast-json-patch
Get the current project pod template:
$ kubectl get pod -o json project-[UUID] > p1.json
$ cp p1.json p2.json
Now, edit p2.json
to your liking, but don’t touch those fields which
are set by Kubernetes! To match the above, e.g. changes like this:
$ diff p1.json p2.json
14c14,15
< "run": "project"
---
> "run": "project",
> "mylabel": "test123"
191,192c192,193
< "serviceAccountName": "default",
---
> "serviceAccountName": "default2",
Now, create a script diff.js
with the following content:
var jsonpatch = require("fast-json-patch");
var fs = require("fs");
var p1 = JSON.parse(fs.readFileSync("p1.json", "utf8"));
var p2 = JSON.parse(fs.readFileSync("p2.json", "utf8"));
var delta = jsonpatch.compare(p1, p2);
console.log(delta);
and run it:
$ node diff.js
[
{
op: 'replace',
path: '/spec/serviceAccountName',
value: 'default2'
},
{ op: 'add', path: '/metadata/labels/mylabel', value: 'test123' }
]
That’s the patch for the license to make the above changes.
Note: under the hood, the patch is stored as a serialized JSON string in the site_licenses database table. e.g.
cocalc=> select * from site_licenses where id = 'f3d76abf-38a5-4e4e-884d-2e70f1afddb9';
id | f3d76abf-38a5-4e4e-884d-2e70f1afddb9
title | patch1
activates | 2023-01-20 16:37:12.595
quota | {"cpu": 1, "ram": 1, "patch": "[{\"op\":\"add\",\"path\":\"/metadata/labels/mylabel\",\"value\":\"test123\"}]"}
run_limit | 1
[...]
Problems?#
To debug this, you have to check the log of manage-action-...
during project startup.
It might fail with an error about a problematic JSON object
(e.g. above, it fails for me, because I do not have a “default2” service account).
You can also set the manage.dbg_project_patching
value to "1"
to
output the project pod JSON right before the patch is applied and also
the patch set itself. Look for lines containing project_pod=
and
project_pod_before=
and patch=
.