Sherin Sunny

Microsoft earnings are out. The company beat expectations, and in a new disclosure, the said revenue from its Microsoft Azure cloud platform surpassed $75 billion for the fiscal year, up 34% from the prior year, driven not just by AI demand but due to “growth across all workloads,” according to a statement from CEO Satya Nadella in the company’s earnings release. Microsoft in the past has bundled Azure revenue in with other cloud services and products for purposes of financial reporting, making it difficult to assess the platform on a standalone basis. With the new disclosure, we have the basis for an almost apples-to-apples cloud revenue comparison, i.e. ... - Google Cloud’s annual revenue run rate has surpassed $50 billion, according to results released last week by its parent company Alphabet. - Amazon, which reports earnings Thursday for the June quarter, reported an annual run rate of more than $111 billion for its AWS cloud business as of the March quarter. I'm still digging through the Microsoft results and will be updating our story here: https://lnkd.in/g6BkT_Bi

205

17 Comments

Cloud in India is moving fast, and I keep seeing how easy it is for teams to get lost in the noise, pricing models, compliance, even just knowing what actually works here. I’ve been digging into this for a while, so we put together a blog on what really matters: cloud adoption trends, cost strategies that don’t break the bank, and navigating India’s regulatory maze. Would love for you to give it a read and tell me where you see the biggest challenges: https://lnkd.in/g7jNxbjt

83

Let's dig into the big Oracle / OpenAI / #Stargate news! This Cloud Wars analysis comprises three main parts: (1) the size and scope of the unprecedented deal, (2) the partial unraveling of OpenAI's once-impregnable @Microsoft relationship, and (3) the deal's implications across the tech landscape. https://lnkd.in/e_dKeqNt

328

8 Comments

Cloud Giants Update: AWS (Amazon): $123B run rate growing 17% YoY (last Q grew 17%) Azure (Microsoft): ~$86B run rate (estimate) growing 39% YoY (last Q grew 35%) Google Cloud (includes GSuite): $54B run rate growing 32% YoY (last Q grew 28%, neither are cc)

96

Spent the afternoon at the Amazon Web Services (AWS) Builder Loft in #SanFrancisco getting a deep dive into #AWSTrainium and the #NeuronSDK — not just how to run models on them, but how the stack works end to end. Trainium is AWS’s custom chip for training and inference, built with multiple NeuronCores, each with its own high-bandwidth memory (HBM). With tensor parallelism, you can split a model across cores to get more compute and memory bandwidth working in parallel — though at some point, communication overhead can offset the gains. The Neuron SDK is the software layer that makes this possible: - Neuron Compiler turns a PyTorch/TensorFlow model into optimized, hardware-specific kernels. - Neuron Runtime loads those kernels onto NeuronCores for execution. - The compiled model is stored as a NEFF file — the binary the hardware actually runs, which can be reused on the same hardware type without recompiling. - Profiling tools let you see latency, utilization, and memory use in real time. We covered how serving choices affect performance just as much as hardware: - Bucketing requests by sequence length so cores aren’t wasting cycles on padding. - Too few buckets = wasted compute, too many = possible latency hit if buckets don’t fill fast enough. - Continuous batching to keep hardware busy between requests — especially effective with steady traffic, but under bursty load it works best when tuned alongside bucket sizes and max sequence length. - Disaggregated prefill to separate prompt processing from decoding, allowing work to be overlapped across cores on long prompts. - PJRT integration so Trainium works with PyTorch XLA, JAX, and vLLM without heavy code changes. - Serving framework integrates cleanly with vLLM, Hugging Face TGI, and Ray Serve, so you can pick the serving layer that matches your latency and concurrency needs. In the workshop, we deployed a Qwen model with vLLM + NeuronX Distributed Inference on an Inf2 instance, tuned tensor parallelism, adjusted bucket configurations, experimented with continuous batching, and watched the changes show up in TTFT, tokens/sec, and utilization. If you’ve worked with Trainium or Neuron SDK, I’d love to hear — what applications are you considering them for, and how are you tuning for your workloads? Thanks to Annapurna Labs for organizing, Ruchi Bhatia for hosting, Kamran Khan and Sindhura Palakodety for helping me get things running smoothly, and Jim Burtoft, Pinak Panigrahi, and Mrinal Shukla for the technical deep dives that connected the hardware and software pieces. #AWS #AWSCloud #AWSTrainium #NeuronSDK #AnnapurnaLabs #vLLM #AI #AIAgents #Amazon

49

2 Comments