When optimizing cloud costs, testing smaller or shared-CPU instances can yield significant savings—especially for always-on, low-traffic services. However, shared CPU instances introduce complexity that can trip you up if you treat CPU counts like hard performance guarantees. Setting appropriate rollback criteria is crucial for safe pilot deployments and maintaining service objectives.
In this post, I will walk you through what has worked running shared CPU tests in AWS and Azure environments using tools like AWS Compute Optimizer and Azure Advisor. I’ll cover why you must consider differences in shared CPU definitions between cloud providers, why averages lie, and how to use meaningful observation windows and percentiles. By the end, you will have clear pilot guardrails and rollback thresholds that protect your service while extracting genuine cost savings.
Always-On Small Services Hide Cloud Waste
Ask yourself: how many small, always-on backend services do you run that get minimal traffic outside business hours? These could be internal APIs, health checks, telemetry forwarders—services that keep ticking but mostly just wait. Running these on big dedicated vCPU machines often wastes cloud budget.
Shared CPU instances promise lower on-demand costs by sharing physical cores among multiple tenants. But this comes with caveats:
- The available CPU time to your instance fluctuates based on noisy neighbors CPU bursting policies vary by provider and instance family Transient spikes can cause throttling that impacts latency and throughput
Therefore, you cannot just eyeball average CPU utilization or total vCPUs from instance specs. You must dig deeper into your workload’s peak CPU needs and how the shared behavior maps to your service objectives. Otherwise, you risk misleading cost optimization attempts that degrade user experience or cause outages.
Shared CPU Definitions Differ By Provider
Before setting rollback thresholds, understand how your cloud provider actually implements shared CPU:
Provider Shared CPU Concept Key Characteristics Relevant Cost Tool AWS CPU Credits and Burstable Instances (e.g., T3, T4g)- Earn CPU credits during idle Spend credits to burst above baseline Credit depletion leads to throttling
- Baseline performance guaranteed Credit system similar but vendor managed Azure Advisor reports VM sizing and cost
Understanding this behind-the-scenes architecture shapes how you interpret CPU utilization metrics. For example, AWS’s CPU credits mean your instance might have nominally spare CPU (0.5 vCPU baseline) but can burst to full vCPU for some time until credits run out. Azure’s burstable VMs provide a baseline CPU guaranteed, but how bursting behaves under sustained load depends on the SKU.
Measure Peaks With The Right Observation Window
Once you know what the cloud provider offers, collect detailed CPU usage metrics. The critical question is: what does the P95 or P99 CPU utilization look like over a window that matches your workload’s burstiness?
Averages hide spikes that cause performance degradation. One approach that I always take is:
Gather high-resolution CPU metrics using your cloud monitoring tools or agent-based telemetry at 1-minute or even sub-minute intervals. Analyze percentiles such as P95 and P99 rather than mean CPU. This tells you how frequently your service experiences CPU pressure that can trigger throttling on shared CPU instances. Choose an observation window that *matches the expected spike duration*. For example, if your app typically has bursts of 2–5 minutes, aggregate your percentiles over rolling 5-minute windows. Shorter or longer windows mask critical temporal patterns. Look also at the duration of high CPU spikes. Are you hitting near 100% for seconds, or sustained minutes? The latter is riskier for shared CPU and should weigh into your rollback planning.This approach helps prevent a classic trap: seeing moderate average CPU utilization (e.g., 40-50%) but occasional spikes that saturate a smaller CPU, causing increased latency or failed requests.
Use Percentiles and Spike Duration, Not Averages
To set realistic rollback criteria and pilot guardrails, quantify what “acceptable” performance looks like. You do this by aligning CPU usage with your service-level objectives (SLOs), such as latency percentiles or error rates, during normal and peak load.
In practice, this means correlating CPU P95/P99 with service performance metrics over a pilot period on your target shared CPU instance type. If performance degrades during observable CPU spikes (e.g., P99 latency increases beyond your SLO), that signals the instance choice is inadequate.
Your rollback thresholds should be based on:
- CPU utilization percentiles: Define maximum acceptable P95 and P99 utilization, ideally below burst thresholds for your instance. Spike length and frequency: Rollback if you observe bursts exceeding defined CPU utilization for longer than a critical duration (e.g., 2 minutes sustained above 90%). Service performance degradation: Rollback if key SLOs breach a threshold during the pilot.
To avoid guesswork, document this as explicit rollback criteria before running tests:
“If P99 CPU utilization exceeds 90% for more than 3 consecutive minutes or error rate goes above 0.1% for 5 minutes, rollback the instance size change immediately.”This kind of guardrail protects you from subtle performance degradations that average CPU or coarse monitoring would miss.
How AWS Compute Optimizer And Azure Advisor Can Guide You
Both AWS and Azure offer advisory services that leverage utilization and configuration data to recommend optimal instance sizes. Neither tool replaces detailed pilot testing, but they provide quality starting points and validation checks.
AWS Compute Optimizer
AWS Compute Optimizer analyzes your historical CloudWatch metrics for CPU, memory, disk, and network. For shared CPU instances like T3 or T4g, it considers CPU credits usage patterns. Its recommendations aim to balance cost and performance risk based on observed peaks, not just averages.
Tip: Use Compute Optimizer’s detailed reports to get predicted utilization percentiles and check if your instance often runs out of CPU credits. This insight forms part of your pilot guardrails and rollback criteria.
Azure Advisor
Azure Advisor provides instance sizing recommendations tuned to your VM’s observed performance and resource use. It highlights underutilized VMs and suggests smaller burstable types when appropriate. Azure Advisor also points out configuration settings or upgrade https://dibz.me/blog/what-should-i-measure-besides-cpu-for-a-shared-cpu-migration-1253 paths that impact cost and performance.
Tip: When migrating to burstable VMs, cross-reference Azure Advisor recommendations with your observed P95/P99 CPU levels. Don’t blindly follow sizing tips without verifying detailed performance characteristics under representative load.
Summary: Setting Rollback Thresholds That Work
To wrap up:
Understand your provider’s shared CPU model. AWS and Azure handle bursting and throttling differently—this shapes your baseline and burst CPU expectations. Measure CPU with a focus on percentiles and spike duration. Don't trust average CPU alone. Align CPU patterns to your service objectives. Know what latency and error rates look like under real load. Define explicit rollback criteria tied to CPU percentiles, spike duration, and service SLO breaches. Run pilots with monitoring and alerting primed to these criteria. If thresholds are hit, rollback early. Leverage AWS Compute Optimizer or Azure Advisor to validate and tune your tests—but conduct your own fine-grained observation and judgment.Ranked among the most common and wasted spend patterns in cloud infrastructure, shared CPU optimization requires discipline and rigor beyond naive assumptions about vCPU counts or averages. Following these steps ensures your small, always-on services run lean and cost-efficiently without degrading user experience.
Your Next Steps
- Review your critical small services and identify candidates for shared CPU testing. Gather high-frequency CPU and latency metrics over at least one week to understand burst behavior. Use AWS Compute Optimizer or Azure Advisor to get initial sizing recommendations. Define your rollback criteria with integrated CPU percentiles and SLOs. Run pilot deployments with automated monitoring and alerts tied to your rollback thresholds. Review pilot data, adjust thresholds as needed, and incrementally roll out changes.
By making these steps part of your cloud cost governance, you move from guesswork to data-driven, safe efficiency improvements.
Always ask: what does P95 and P99 CPU utilization look like before touching instance types? And write your rollback https://bizzmarkblog.com/are-bots-and-internal-services-good-on-shared-cpu-if-concurrency-is-low/ criteria before starting the pilot.
Have you run shared CPU pilots with rollbacks in your cloud environment? What tools and thresholds worked well for you? Comments and questions welcome!