Generative AI & Large Language Models

What this area actually covers

This area is the engineering of product features whose behaviour is produced by a language model rather than by code you wrote. You define what the feature should do, you construct the input that will make a model do it, you constrain the output enough that the rest of your system can consume it, you measure whether the whole thing is any good, and you run it inside a latency budget and a cost budget that somebody will hold you to. The model itself is a dependency, usually one you did not train and cannot inspect.

That framing is deliberate, because it locates the difficulty correctly. Almost nobody working in this area is training a foundation model, and the small number who are belong to a different discipline with different prerequisites. What the majority do is substantially closer to integration engineering with an unusual dependency: a dependency that is probabilistic, whose behaviour changes when the provider updates it, whose failures are fluent and plausible rather than loud, and which will do whatever the text flowing through it tells it to do. Every hard problem in this section descends from one of those four properties.

The boundary with adjacent areas is worth drawing now, because the vocabulary overlaps and the interviews do not. Retrieval — how you find the right documents to put in front of the model — is a large enough engineering surface that Preptima keeps it separate in RAG, Retrieval & Agents, and in a real system it is usually where most of your effort goes. Training models from data you own, choosing between gradient boosting and a neural network, and reasoning about leakage and class imbalance belong to Machine Learning. The infrastructure for versioning, deploying, monitoring and paying for models is MLOps & ML Platform, and while the LLMOps subsection here covers what is specific to language models, the general discipline lives there.

There is also a housekeeping point about this site. The older Data & AI Engineering section still carries an LLMs & RAG subsection alongside MLOps, ML Fundamentals and Statistics & Experimentation from earlier authoring waves. Those pages have not been removed and are still worth reading; they are simply a single-subsection treatment of material that has since been given four sections of its own. If you find the same topic in both places, the version here is the more detailed one and the older one is the compressed introduction.

Three things get wrongly bundled into this area. Prompt engineering as a standalone job is one — prompting is a technique inside the work, not the work, and a role defined by it alone has not survived contact with hiring. Data science is another: the instinct to treat an LLM feature as a modelling problem with a train and test split leads people to skip the part where a product decision has to be made about what "correct" means. And model research is the third; reading papers about architectures is interesting and is almost entirely orthogonal to whether your summarisation feature returns parseable JSON under load.