Create a code assistant with Continue and AI Endpoints

---

If you want to have more information on AI Endpoints, please read the following blog post.
You can, also, have a look at our previous blog posts on how use AI Endpoints.
Also, be sure to check out this blog post about deploying Coder on the OVHcloud Managed Kubernetes Service.

In the field of the code assistant a lot of solutions exist.

However, you should want to create your own assistant to master your configuration and to have a better control on your data.

Continue will help you in this task.

In a nutshell, Continue is an IDE plugin to build your own code assistant.

At the time of writing this blog post, Continue is compatible with VSCode and JetBrains IDE.

The great strength of Continue is that it allows to use a custom LLM endpoints and, yes, AI Endpoints works with Continue 😎.

Continue installation

Continue come as a plugin, it’s very simple to install, please follow the official documentation.

Note that when you install Continue for one IDE, the settings are shared for the other IDE.

Continue settings with AI Endpoints

Once continue is installed, you can set it up with AI Endpoints.

Continue has two major configurations, one for the chatbot tool and another one for the tab completion tool.

To add the configurations, open the JSON configuration file and set it up as follow:

{
  "tabAutocompleteModel": {
    "title": "CodeLlama-13b completion",
    "model": "CodeLlama-13b-Instruct-hf",
    "apiBase": "https://codellama-13b-instruct-hf.endpoints.kepler.ai.cloud.ovh.net/api/openai_compat/v1",
    "provider": "openai",
    "useLegacyCompletionsEndpoint": true,
    "apiKey": "<your API key>"
  },
  "models": [
    {
      "title": "LLaMa 3 70B",
      "model": "Meta-Llama-3-70B-Instruct",
      "apiBase": "https://llama-3-70b-instruct.endpoints.kepler.ai.cloud.ovh.net/api/openai_compat/v1",
      "provider": "openai",
      "useLegacyCompletionsEndpoint": false,
      "apiKey": "<your API key>"
    },
    {
      "title": "CodeLlama-13b completion",
      "model": "CodeLlama-13b completion",
      "apiBase": "https://codellama-13b-instruct-hf.endpoints.kepler.ai.cloud.ovh.net/api/openai_compat/v1",
      "provider": "openai",
      "useLegacyCompletionsEndpoint": false,
      "apiKey": "<your API key>"
    }
  ] 
  // ... 
}

As you see you can define only one model for the tab completion feature, we advise you to use CodeLlama for this.

For the chatbot tool you can set several models, try them and choose the one that fit the best to your needs 😊.

It’s time to test it!

The chatbot tool.

The tab completion tool.

Don’t hesitate to test our new product, AI Endpoints, and give us your feedback.

You have a dedicated Discord channel (#ai-endpoints) on our Discord server (https://discord.gg/ovhcloud), see you there!