Language Models

Note

This was added in version 2.15.0

It’s now possible to run the entire range of supported Large Language Models on CoCalc Cloud. You can configure this like all other settings in the “Admin panel” → “Site settings”. Click on AI-LLM to filter the configuration values.

On top of that, you can limit the choices for your users by only included selected models in list called User Selectable LLMs.

In particular, running your own Ollama server and registering it might be an interesting option.

Note

Just like with all the other admin settings, you can also control their values via your my-values.yaml, in the globals.settings dict.

Ollama

To tell CoCalc Cloud to talk to your own Ollama server, you have to set the ollama_configuration site setting.

Assuming your Ollama server runs at the host and port ollama.local:11434, you have to set this in global.software to access a model named gemma:

ollama_configuration: '{"gemma" : {"baseUrl": "http://ollama.local:11434/" , cocalc: {display: "Gemma", desc: "Google''s Gemma Model"}}}'

You can add more models by adding more dictionary entries.