Building Amália - what it takes to put a face on a public LLM

The Portuguese government did something rare this year. It funded and released an open-source large language model trained for European Portuguese, weights and all, free for anyone to run. Then it stopped there. There was a model on Hugging Face and a white paper, but no way for a normal person to actually talk to it. You needed a GPU, a serving stack, and a tolerance for command lines.

That gap is the whole reason Amália Chat exists. We wanted a Portuguese speaker to open a link, type a question in their own language, and get an answer. The kind of thing people already know how to use because it looks like every other chat app they've tried.

This is the story of how we got there, feature by feature, and the decisions that turned out to matter more than we expected.

Starting boring on purpose

The first hours were deliberately unglamorous. We scaffolded a Next.js app on the App Router with Tailwind, wired up Postgres, and put Better Auth in front of it. Streaming came next, through the Vercel AI SDK talking to Amália over an OpenAI-compatible endpoint. Then conversation persistence, a sidebar to switch between chats, and Markdown rendering so code and lists looked right.

None of that is a headline. But it's the floor everything else stands on, and getting it solid early meant the interesting features later didn't have to fight the foundation.

One choice from this phase paid off repeatedly: passwordless auth with guest sessions. A first-time visitor doesn't sign up. They just start typing. Guests get a few free messages, counted server-side, and when they hit the limit the app turns an HTTP 402 into a friendly signup wall. If they do sign up with an email code, we link their anonymous session to the new account and carry their existing conversation over with them. Nobody loses their chat because they decided to register halfway through.

That path did real work. In the first twelve days about 3,300 people hit the free-message wall, and roughly 980 of them created an account rather than leaving. A wall that converts close to a third of the people who reach it is a wall people apparently didn't mind.

Planning for the day it gets popular

A model like this attracts a spike. Someone posts the link, a newspaper picks it up, and suddenly a few thousand people arrive at once. Our backend has a fixed number of generation slots, so the honest question was: what happens when demand exceeds them?

The answer we built is a chain of graceful failures instead of one hard crash.

At most a set number of generations run at the same time. Everything past that waits in a FIFO queue, and here's the part users actually feel: their live position in that queue streams down to the browser, so they see "3rd in line" tick toward the front instead of staring at a frozen spinner. When even the queue is full, the request comes back as a clean "at capacity" state, and we offer an email waitlist so the person can leave their address and come back later.

Queued requests don't hang forever; they give up after a bounded wait. Rate-limit checks happen before we reserve a slot, and we only save the user's message after they've been admitted, so a request that gets turned away or abandoned costs us nothing. There's even a throughput knob: shorten the maximum reply length under heavy load to turn slots over faster, then loosen it again when things calm down.

We hoped never to need most of this. Building it anyway meant a busy launch would bend, not break.

And the launch came. Day one brought 2,318 visitors. Day two brought 4,550, nearly seven thousand people in the first 48 hours, all arriving against a fixed pool of generation slots. The traffic held up, then settled into a steady rhythm of a couple hundred a day. The week we spent on queues and load shedding paid for itself in the first two.

Two boxes, and keeping their caches warm

Early on we ran everything through one endpoint. Then we brought a second machine online, a llama.cpp server on a Mac alongside the one on the NVIDIA DGX Spark, and hit a subtle problem.

Our endpoint config already accepted a comma-separated list of backends, but it round-robined every single generation. That meant two consecutive turns of the same conversation could land on different machines, and each one would reprocess the whole prompt from cold. All that prefix cache the first machine had built up went to waste.

The fix was to stop spreading generations and start spreading conversations. We hash each conversation's id and map it to one stable backend, so every turn of that chat goes to the same box and its cache stays warm across the whole exchange. Different conversations still fan out across machines, so we keep the load balancing. And if you want to lean traffic toward a faster box, you just list its URL twice. No code change, no config gymnastics.

Somewhere in here we also started recording which model produced each assistant message, so as we moved from the fine-tuned checkpoint to the preference-tuned one we could tell, message by message, where each answer came from. Small thing. Very useful when you're debugging quality later.

Letting people share what they found

Once conversations were persistent, the natural next ask was to show them to someone else. A good answer from Amália is something you want to send to a friend or drop in a group chat.

Share links are trickier than they look, and most of the work was in the boundaries. Each conversation can get an unguessable share token. When you share, we stamp a "shared through" timestamp using the database clock, and the public page only ever renders messages up to that moment. So if you keep chatting after sharing, your later messages stay private; the link is a frozen snapshot, not a live window into your ongoing conversation.

Only registered owners can create a link, guests can't. Re-sharing after you've revoked a link mints a fresh token, so the old URL stays dead. Empty conversations refuse to be shared at all. The public page itself is a read-only server component marked noindex, so shared links don't leak into search results. Revoke it and the conversation goes private again, instantly.

Being honest about what the model is

A model trained by a research effort, released with a white paper, comes with real limits. It has a knowledge cutoff. It can be wrong. And LayerX only built the interface; we didn't train the model and we're not the authority on its answers.

So we added a Terms link in the composer footer that opens a modal laying all of this out in European Portuguese. It explains the knowledge cutoff plainly, that Amália has no live internet access, that it can make mistakes, and where our responsibility as the interface ends and the model's behavior begins. We grounded every claim in the actual white paper rather than inventing a cutoff date, because none is published and we weren't going to assert one we couldn't back up.

It's a small modal. It also felt like the least we owed people who trust the thing.

Instrumenting the backend

By this point the app was doing enough that "check the logs" needed to mean something. We put Pino across the whole backend: pretty-printed in development, structured NDJSON to stdout in production, full stack traces on errors.

The request pipeline now carries a per-request logger tagged with request, user, and conversation ids, so you can trace a single chat turn end to end. We log when a generation starts and which backend it hit, when it finishes with token counts and latency, when the queue admits or sheds load, and when the guest-to-account migration runs. The rule we held to: never log the OTP code, never log raw emails, never log the raw search query. Observability shouldn't become a privacy leak.

There's also an ASCII-art startup banner now, printed once per boot. Purely for the people running it. Sometimes you build a small nice thing because it makes the work feel like yours.

Cleaning up and searching the web

Two of the most recent additions round out the experience.

Deleting a chat sounds trivial until you delete the wrong one. We routed every delete, whether from the header's three-dot menu or the sidebar trash icon, through a shared confirmation dialog. One reusable destructive-action modal, one clear "are you sure", consistent everywhere.

And then web search, which is the feature I'm most excited about. Amália can now search the web mid-reply through a Tavily-backed tool. Ask it something current and it can go look, then answer with what it found, instead of guessing from training data that has a cutoff.

We rolled this out carefully. It's off by default and gated behind a per-user allowlist, checked on the server where it can't be spoofed. A menu button beside the composer opens an options popover with a web-search toggle. Allowlisted accounts get it with a "Beta" badge; registered users who aren't invited yet see it gated as invite-only, and signed-out visitors are told to register first. The toggle state lives high enough in the app that it survives switching between conversations, and the flag rides along with each message, though the server re-checks the allowlist every time regardless of what the client claims.

There were sharp edges to file down. We bound the Tavily fetch with a timeout so a hung upstream can't pin a generation slot for the full route duration. We keep the "writing" indicator visible while a search runs, since the assistant turn has no text yet during the lookup. And we made sure assistant text gets saved across all tool steps, not just the final one, so nothing said before a search call gets dropped.

The first twelve days, in numbers

The analytics tell a clearer story than any feature list.

• Just under 12,000 people used Amália, sending around 35,000 messages and getting back nearly 36,000 replies.
• 73% of them were in Portugal, which is exactly who the model was built for. The rest came mostly from the US, the Netherlands, Ireland, and Spain.
• People stayed. The average visit ran close to six minutes, and only 28% bounced, high engagement for a tool nobody had heard of two weeks earlier.
• Signing up changed how people used it. Guests sent about 13 messages each before drifting off; the people who created an account sent closer to 36. The wall did its job, and the accounts behind it were worth more than the free traffic.
• The concise-by-default instruction we gave the model shows up in the data. Roughly six in ten replies came back under 512 tokens, and around 86% stayed under 2,000. Amália answers and stops, which is the behavior we asked for.
• Web search and share links are new enough that their numbers are tiny, and that's fine. Web search in particular is still invite-only and barely switched on. They're the features that will matter more as the daily regulars settle in than they did in the launch rush.

What building this taught us

The features people notice are share links and web search. The features that actually keep Amália standing are the queue, the cache pinning, and the boring foundation from week one.

Putting a face on a public model turned out to be less about the model and more about everything around it: gracefully running out of capacity, being honest about limits, letting people leave and come back, keeping a shared link from saying more than its owner meant. The government shipped the hard part. Our job was to make it feel like something a person could just use.

It's live, it's in Portuguese, and it works. That was the whole point.