You built the algorithms by hand - the update rule, the sigmoid, the split. Now a decision that has nothing to do with gradient descent and everything to do with not wasting six months: for THIS task, do you train a classical model, fine-tune a foundation model, or just prompt one? The wrong choice is expensive. A team that fine-tunes a large language model to tag support tickets may spend weeks and thousands of dollars to ship something a logistic regression would have beaten in an afternoon. Another team prompts a model for every fraud check on millions of transactions and watches the cloud bill dwarf the fraud it catches. Three tools, one job, and the skill is knowing which to reach for. This lesson gives you a framework for that choice on the three axes that actually decide it: data, cost, and latency.
flowchart TD Q["Start: how much<br/>labeled data"] --> A["under 50 rows<br/>prompt a model"] Q --> B["hundreds to thousands<br/>fine-tune or classical"] Q --> C["50k or more<br/>train classical"] style C fill:#10b981,color:#fff style A fill:#b45309,color:#fff
Three tools, three cost structures
A classical model - logistic regression, decision trees, gradient boosting - is cheap to train, cheap to serve, and scores in microseconds. It asks for hundreds to thousands of labeled rows and shines on tabular, structured data: numbers and categories in columns. Fine-tuning takes a foundation model that already understands language or images and adapts it to your task with a few hundred to a few thousand labeled examples. Training burns GPU hours, so you pay a real upfront cost, but each prediction afterwards is far cheaper than prompting. Prompting trains nothing: you send the task and the model answers. Zero upfront cost, but you pay the full price on every single call, and every call waits on a network round-trip to the model. The choice is really about where the cost lands - upfront, or per call - and how much of the work your data can do for you.
def monthly_cost_mills(approach, calls):
if approach == "prompt":
return 5 * calls # 5 mills per call, no training
if approach == "finetune":
return 200000 + calls # 200000 mill training + 1 mill per call
return 5000 + calls // 100 # classical: 5000 mill training, ~0 per call
for a in ("prompt", "finetune", "classical"):
print(a, monthly_cost_mills(a, 50000))
Read the numbers, then generalize
At 50,000 calls a month, prompting and fine-tuning cost the same - 250,000 mills, or $250. That is the breakeven point: below it, prompting wins because fine-tuning's training cost is not yet paid back; above it, fine-tuning wins because its cheaper per-call rate overtakes the upfront fee. Classical, meanwhile, sits at $5.50 regardless, because its per-call cost is effectively zero. This is the whole decision on one axis: where does your call volume sit relative to the breakeven, and is your data tabular enough that a classical model is even an option? The exact dollar figures move with the model and the provider, but the shape - a fixed cost amortizing against a per-call cost - is permanent.
flowchart LR A["at zero calls<br/>prompt and finetune tie on per-call"] -->|"each extra call"| B["prompt adds 5 mills<br/>finetune adds 1 mill"] B --> C["finetune pays back<br/>its 200000 mill training"] C --> D["breakeven at<br/>50000 calls a month"] style D fill:#166534,color:#fff
The data axis: how much do you have?
How much labeled data you have rules out options before cost ever comes up. With under a few dozen labeled examples you cannot reliably train or fine-tune anything - there is too little signal. Prompting wins by default, because the foundation model brings its own knowledge and needs none of your data. With hundreds to a few thousand rows, fine-tuning comes into reach: enough signal to adapt the model, not enough to justify training a large classical model from scratch. With tens of thousands of labeled rows of tabular data, a classical model usually wins outright - it has the examples it needs, it trains in seconds, and it serves for fractions of a cent. Notice that volume alone does not pick the tool; volume AND data type together do. A million rows of free text still lean on the model that already speaks the language.
def serve_latency_ms(approach):
if approach == "classical":
return 1 # local scoring
if approach == "finetune":
return 80 # self-hosted model inference
return 300 # prompt: round-trip to a hosted model
for a in ("classical", "finetune", "prompt"):
print(a, serve_latency_ms(a))
Latency: the third axis
Cost and data are not the whole story. A model behind an API that adds 300ms to every request is fine for a batch job that runs overnight and fatal for an autocomplete that fires on every keystroke. A classical model scoring in a millisecond can sit directly in the request path of a high-traffic feature; a prompted model usually cannot. Latency rarely picks the tool on its own, but it vetoes: if your service-level budget is 20ms, prompting is off the table no matter how cheap or accurate it is. Ask the latency question early, because it is the one constraint you cannot engineer away by spending more - the speed of light and a hosted model's queue are what they are.
What does this print? Costs are in mills; fine-tuning has a 200000 mill training fee plus 1 mill per call.
def monthly_cost_mills(approach, calls):
if approach == "prompt":
return 5 * calls
if approach == "finetune":
return 200000 + calls
return 5000 + calls // 100
print(monthly_cost_mills("prompt", 50000))
print(monthly_cost_mills("finetune", 50000))
print(monthly_cost_mills("classical", 50000))
prompt = 5 * 50000 = 250000.
finetune = 200000 + 50000 = 250000.
classical = 5000 + (50000 // 100) = 5500.
Write breakeven_calls(a_fixed, a_per_call, b_fixed, b_per_call) returning the call count n at which approach A and B cost the same. Costs are in mills; fixed is a one-time cost and per_call is mills per call. Solve a_fixed + a_per_call*n == b_fixed + b_per_call*n for n. The inputs are chosen so n divides evenly - use integer division.
def breakeven_calls(a_fixed, a_per_call, b_fixed, b_per_call):
# solve a_fixed + a_per_call*n == b_fixed + b_per_call*n for n
pass
Rearrange: n = (b_fixed - a_fixed) / (a_per_call - b_per_call).
The per-call rates differ, so the denominator is non-zero; use integer division.
def breakeven_calls(a_fixed, a_per_call, b_fixed, b_per_call):
return (b_fixed - a_fixed) // (a_per_call - b_per_call)
cheaper(approach_a, approach_b, calls) should return the LESS expensive approach, but the comparison runs the wrong way and returns the more expensive one. Fix the one operator. (monthly_cost_mills is correct and is provided.)
def monthly_cost_mills(approach, calls):
if approach == "prompt":
return 5 * calls
if approach == "finetune":
return 200000 + calls
return 5000 + calls // 100
def cheaper(approach_a, approach_b, calls):
if monthly_cost_mills(approach_a, calls) > monthly_cost_mills(approach_b, calls):
return approach_a
return approach_b
To return the cheaper of the two, approach_a must be chosen when its cost is LOWER than b's.
The operator should be less-than, not greater-than.
def monthly_cost_mills(approach, calls):
if approach == "prompt":
return 5 * calls
if approach == "finetune":
return 200000 + calls
return 5000 + calls // 100
def cheaper(approach_a, approach_b, calls):
if monthly_cost_mills(approach_a, calls) < monthly_cost_mills(approach_b, calls):
return approach_a
return approach_b
You have 80,000 labeled rows of loan-application data (income, credit score, debt ratio - pure tabular), you predict default yes/no, and you serve 5 million decisions a month under a 20ms p99 latency budget. Which approach is the strongest FIRST choice?
Tabular data, very high volume, and a tight latency budget all point to a classical model: it trains fast, serves in microseconds for fractions of a cent, and beats neural alternatives on structured columns. Prompting 5M times a month would be slow and expensive; fine-tuning an LLM buys nothing on tabular features.
Now route support tickets into your company's 47 internal product categories - names only your company uses. You have 1,200 labeled tickets and route 30,000 a month. What is the strongest argument for FINE-TUNING over prompting here?
The label space is company-specific, so a general prompt guesses at categories it was never taught. 1,200 examples is enough to fine-tune, and 30,000 monthly calls amortizes the training cost. Fine-tuning is not always cheaper, does need labeled data, and never removes the need to evaluate.
Recap
- Three tools: a classical model (cheap, fast, tabular), fine-tuning (adapts a foundation model with hundreds-thousands of examples), and prompting (no training, pay per call).
- The data axis rules out options first: almost none means prompt; a medium amount means fine-tune or classical; a large tabular set means classical.
- On cost, fine-tuning overtakes prompting past a breakeven volume; classical is cheapest at almost any volume when it fits the data.
- Latency vetoes: a 20ms budget kills prompting no matter how cheap or accurate.
- The classic trap is fine-tuning when a classical model or a plain prompt would already meet the bar - reach for it last, not first.
Next you will follow a model out of the notebook and into production, where latency, cost, and drift decide whether it survives contact with real users.