Full-Stack AI: Fastify, Deno Deploy, DigitalOcean, and LM Studio Integration

Building a Full-Stack AI Application with Fastify, Deno Deploy, DigitalOcean, and LM Studio

The modern web development landscape demands high-performance, scalable, and intelligent applications. Integrating a robust backend framework with serverless deployment and local Large Language Models (LLMs) offers a powerful solution. This article demonstrates how to combine Fastify for API development, Deno Deploy for serverless hosting, DigitalOcean for infrastructure, and the LM Studio API for local LLM inference, all powered by TypeScript.

Architecture Overview

Our setup involves a Fastify API deployed on Deno Deploy, acting as the frontend for user requests. This API will then communicate with an LM Studio instance, conceptually hosted on a DigitalOcean Droplet, to perform LLM inference. This decouples the compute-intensive LLM from the highly scalable Deno Deploy frontend.

Fastify on Deno: The API Gateway

Fastify is a highly performant, low-overhead web framework for Node.js and Deno. Its plugin-based architecture and robust validation make it ideal for building efficient APIs. Deno's native TypeScript support and secure runtime perfectly complement Fastify.

First, ensure you have Deno installed. Create a main.ts file:

import Fastify from "https://deno.land/x/fastify@v4.26.2/mod.ts";
 
const app = Fastify({ logger: true });
 
app.get("/", async (request, reply) => {
  return { hello: "world" };
});
 
app.listen({ port: 8000 }, (err, address) => {
  if (err) {
    app.log.error(err);
    Deno.exit(1);
  }
  console.log(`Server listening on ${address}`);
});

This basic Fastify server runs on Deno. Note the direct URL import for Fastify, a Deno-specific feature.

Integrating with LM Studio API (via DigitalOcean)

LM Studio allows you to run local LLMs via an OpenAI-compatible API. While LM Studio typically runs on a local machine, for a production setup, you'd host it on a dedicated server. A DigitalOcean Droplet with sufficient GPU resources is an excellent choice for this, providing a stable, accessible endpoint for your LLM.

Let's extend our Fastify app to call an external LLM endpoint. We'll assume your LM Studio instance is running at http://YOUR_DIGITALOCEAN_IP:1234/v1/chat/completions.

// main.ts (continued)
import Fastify from "https://deno.land/x/fastify@v4.26.2/mod.ts";
 
// ... (previous Fastify setup)
 
app.post("/chat", async (request, reply) => {
  const { prompt } = request.body as { prompt: string };
  if (!prompt) {
    return reply.status(400).send({ error: "Prompt is required" });
  }
 
  try {
    const lmStudioApiUrl = Deno.env.get("LM_STUDIO_API_URL") || "http://localhost:1234/v1/chat/completions";
    const response = await fetch(lmStudioApiUrl, {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        // "Authorization": "Bearer YOUR_API_KEY" // If your LM Studio instance requires auth
      },
      body: JSON.stringify({
        messages: [{ role: "user", content: prompt }],
        model: "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q4_K_M.gguf", // Or your chosen model
        temperature: 0.7,
        max_tokens: 150,
      }),
    });
 
    if (!response.ok) {
      const errorText = await response.text();
      app.log.error(`LM Studio API error: ${response.status} - ${errorText}`);
      return reply.status(response.status).send({ error: "Failed to get response from LLM" });
    }
 
    const data = await response.json();
    return reply.send(data.choices[0].message.content);
  } catch (error) {
    app.log.error(`Error calling LM Studio API: ${error}`);
    return reply.status(500).send({ error: "Internal server error" });
  }
});
 
// ... (app.listen)

This code snippet demonstrates a /chat endpoint that takes a prompt, forwards it to the configured LM Studio API endpoint, and returns the LLM's response. We use Deno.env.get for the API URL, a crucial best practice for secure configuration.

Deploying to Deno Deploy

Deno Deploy provides a global, low-latency edge network for Deno applications. It automatically detects and deploys Deno projects directly from a Git repository, offering a seamless serverless experience.

1. Initialize Git: git init && git add . && git commit -m "initial commit"
2. Create GitHub Repository: Push your code to a new GitHub repository.
3. Connect Deno Deploy: Go to Deno Deploy Dashboard, create a new project, and connect it to your GitHub repository. Select main.ts as your entry point.
4. Environment Variables: Add LM_STUDIO_API_URL to your Deno Deploy project's environment variables, pointing to your DigitalOcean Droplet's LM Studio API endpoint.

Deno Deploy will automatically build and deploy your application, making it accessible via a public URL.

DigitalOcean for LM Studio Hosting

While not directly part of the Deno Deploy application, a DigitalOcean Droplet is the recommended way to host your LM Studio instance for production. You would provision a Droplet with a compatible GPU (if needed for performance), install LM Studio, load your desired models, and expose its API securely. This provides the dedicated resources and network accessibility required for your Deno Deploy application to interact with it.

Conclusion

By integrating Fastify on Deno Deploy with a DigitalOcean-hosted LM Studio API, you create a powerful, scalable, and cost-effective AI application. This architecture leverages Fastify's performance, Deno Deploy's serverless efficiency, and LM Studio's local LLM capabilities, all orchestrated with TypeScript for robust development. This pattern offers a flexible foundation for building intelligent web services that can adapt to evolving AI models and infrastructure needs.