Learning by Patrik

Optimize generative AI model performance with Microsoft Foundry | AI-103 | Episode 5

Getting better results from a generative AI model does not automatically mean fine-tuning it. The key is identifying what is wrong with the output and choosing the least complex technique that solves it.

The decision you should remember

Problem	Technique	Why
Instructions, tone or output format	Prompt engineering	Fastest and simplest
Missing or external knowledge	RAG	Retrieves relevant information at runtime
Consistently wrong behavior/style	Fine-tuning	Changes how the model responds
Knowledge + behavior problems	RAG + fine-tuning	Both may be required

Start with prompt engineering. System instructions can define the model's role, constraints, tone and expected output. Examples and few-shot prompting can further improve consistency.

RAG = give the model knowledge

Retrieval-Augmented Generation is appropriate when the required information is too large, specialized or dynamic to include directly in the prompt:

Question → Vectorize → Retrieve relevant chunks → Question + context → LLM → Answer

The important distinction is that RAG does not retrain the model. Relevant external information is retrieved and added to the model's context at runtime.

Fine-tuning = change behavior

Fine-tuning trains a base model using many examples of desired input/output behavior. A supervised dataset commonly contains conversations such as:

{"messages":[
  {"role":"user","content":"Suggest a destination."},
  {"role":"assistant","content":"Absolutely! What kind of trip interests you?"}
]}

Microsoft Foundry supports different customization methods depending on the selected model, including supervised fine-tuning and Direct Preference Optimization (DPO). Model and region must support the chosen method.

Key distinction: Fine-tuning is comparatively expensive and time-consuming, produces a new fixed model that must be deployed, and must be repeated when training requirements change.