Every cloud instance you launch — from a tiny t2.micro to an eight-GPU training beast — is really just a slice of a physical server carved up by a hypervisor (AWS uses KVM; older instance generations used Xen), which partitions one physical machine's CPU, memory, network, and storage into isolated VMs by oversubscribing capacity across tenants. That oversubscription is exactly why providers can offer steep discounts on capacity they don't expect everyone to claim at once: Spot pricing.
On-Demand pricing is your baseline: a fixed hourly rate, no commitment, no interruption risk — the right default while prototyping or running anything that must not be interrupted mid-job. Spot Instances let you bid for AWS's unused capacity at a 50–90% discount, but AWS can reclaim that capacity with only a 2-minute interruption warning. Spot is a great fit for fault-tolerant workloads — batch jobs, hyperparameter sweeps, checkpointing training runs — and a poor fit for anything that must run to completion uninterrupted, like production inference. Check the EC2 Spot Instance Advisor before committing to a Spot strategy: it shows historical interruption-frequency ranges and current pricing by instance type and region.
For GPU workloads, instance family matters as much as pricing model: G5 (NVIDIA A10G) for everyday inference and light fine-tuning (~\$1.21/hr on-demand, 60–70% Spot savings); P5 (NVIDIA H100, 80GB) for large-scale training, up to 4x faster time-to-solution than the prior generation; Trainium3 (AWS custom silicon) for ~50% lower training cost when your toolchain can adapt.
Two operational traps to internalize. First: stopping an instance stops compute billing, but not storage billing — the attached EBS volume is billed separately by the GB-month and keeps accruing charges until you explicitly delete it, not just stop the instance. Terminate an instance (which deletes its root EBS volume by default) when you're truly done, or explicitly delete unused volumes. Second: if you use Spot for training, design for interruption from the start — catch the 2-minute warning and checkpoint model state (weights, optimizer state, step/epoch) to durable storage before the instance disappears. Checkpoint too rarely and an interruption near the end of a long run wipes out hours of progress; checkpoint too often and the I/O overhead becomes a bottleneck. The right frequency depends on model size and storage write throughput, tuned per job.
That durable storage destination is almost always object storage — on AWS, Amazon S3. Unlike your laptop's hierarchical folders, S3 stores objects (a blob of data plus metadata) inside a flat bucket; what looks like datasets/wine-quality/train.csv is really a single object whose key happens to contain slashes. No directory tree to traverse is part of why S3 offers 99.999999999% (11 nines) durability at near-limitless scale.
Not all data is accessed equally often, so S3 storage classes let you match cost to access pattern: Standard (active datasets, millisecond access, no fee), Standard-IA (infrequent access, small retrieval fee), and Glacier (long-term archives, minutes-to-hours retrieval, lowest storage cost). Lifecycle rules automate the migration — e.g., "after 30 days move to Standard-IA, after 90 days move to Glacier" — with no code and no engineer remembering to do it.
Versioning keeps every prior copy of an object when it's overwritten or deleted, which saves you when a bad script clobbers your training data. But versioning without a lifecycle partner is the classic rookie mistake: every old version sticks around forever and the bill grows unbounded. Always pair versioning with a lifecycle rule that expires noncurrent versions after a defined window.
S3 buckets are private by default. Public exposure always requires an explicit action — an overly permissive bucket policy, a legacy ACL granting access to "Everyone," or someone disabling Block Public Access. Headline "misconfigured S3 bucket" breaches are really breaches from someone removing protections that were on by default; leave Block Public Access enabled unless you have a specific, reviewed reason to serve public content, and scope any exception as narrowly as possible. For encryption, S3 offers SSE-S3 (AWS-managed keys, automatic) and SSE-KMS (customer-managed keys with auditable access control) at rest, plus TLS-enforced transit — you can even add a bucket policy condition that denies any request where aws:SecureTransport is false.
One CLI distinction worth knowing: aws s3 cp copies files every time you run it, while aws s3 sync compares source and destination and only transfers what's new or changed — far more efficient for repeated uploads of large datasets.