Before we start, one clarification that trips people up: “LLM architecture” means two different things depending on who’s asking. There’s the model architecture the neural network itself (the transformer). And there’s the system architecture everything you bolt around a model to ship a product (retrieval, orchestration, guardrails, APIs). This post is about the first one: what’s inside the model. Once you understand that, the system stuff makes a lot more sense too.

The Short Answer

Modern LLMs are decoder-only transformers. Text is broken into tokens, each token becomes a vector (embedding), and those vectors flow through a stack of identical transformer blocks. Each block does two things: an attention step that lets every token look at the tokens before it, and a feed-forward step that transforms the result. After the final block, the model outputs a probability distribution over the next token. Generation is just doing this over and over, one token at a time. That’s it, the intelligence is an emergent property of a very large, very well-trained version of this simple loop.

Step 1: Text Becomes Numbers

A neural network can’t operate on characters, so the first job is turning text into numbers.
Tokenization splits your input into tokens usually subword chunks, not whole words. The word tokenization might become token + ization, while common words like the are a single token. Most models use a scheme like Byte-Pair Encoding (BPE) that balances vocabulary size against sequence length. A vocabulary of roughly 30,000 to 200,000 tokens is typical.

You can try GPT tokenzier here.

Each token maps to an ID, and each ID maps to a learned embedding a vector of, say, a few thousand numbers that encodes the token’s meaning. Embeddings are learned during training, so semantically related tokens end up near each other in vector space.

There’s one catch: attention (coming up next) has no inherent sense of order it treats a sequence like a bag of tokens. So we inject position information. Older models added fixed or learned positional encodings; most current models use Rotary Position Embeddings (RoPE), which encode position by rotating the query and key vectors. The practical upshot is the same: the model knows that “dog bites man” and “man bites dog” are different.

Step 2: Attention – The Core Idea

Self attention is the mechanism that made transformers work, and it’s the one piece worth understanding deeply.

For every token, the model computes three vectors by multiplying its embedding by learned weight matrices:

• Query (Q): what this token is looking for.
• Key (K): what this token offers to others.
• Value (V): the actual information this token passes along.

To decide how much token A should pay attention to token B, you take the dot product of A’s query with B’s key. High dot product means “relevant.” Those scores are scaled, passed through a softmax to turn them into weights that sum to 1, and used to build a weighted sum of the value vectors. In compact form:

Attention(Q, K, V) = softmax( (Q · Kᵀ) / √dₖ ) · V

That single equation is the heart of every LLM. It lets the word it in “the cup fell off the table and it broke” actually connect back to cup.

Two refinements make it powerful in practice:

• Multi-head attention. Instead of one attention calculation, the model runs several in parallel (each a “head”), each free to focus on a different kind of relationship grammar, coreference, topic. Their outputs are concatenated and mixed.
• Causal masking. In a decoder-only model, a token may only attend to tokens before it, never ahead. This is enforced by masking future positions with negative infinity before the softmax. It’s what makes the model able to generate text left to right without cheating.

A single attention head is one way of asking: “for each word, which other words should I look at?”

Why multiple heads

One head can only learn one kind of relationship. In “The cat that chased the mouse was hungry,” you need to know:

• was – agrees with cat (syntax)
• chased – links cat and mouse (who did what)
• hungry – describes cat (semantics)

One head averaging all of this gets mush. So you run h heads in parallel, each with its own small Q/K/V projections, each free to specialize. Then you concatenate their outputs and pass them through one final linear layer to mix them back into a single vector.

The trick: each head works in a smaller dimension, so 8 heads cost roughly the same as 1 big head. You get diversity for free.

Quick analogy: a panel of h readers all read the same sentence. One tracks grammar, one tracks who-did-what-to-whom, one tracks pronoun references. Each writes a short note. An editor merges the notes into one summary. More readers with different specialties beats one reader trying to track everything at once.

Causal masking

Causal (autoregressive) masking stops a token from attending to anything that comes after it otherwise the model would cheat during training by peeking at the answer it’s supposed to predict.

Each row is one token asking “who should I look at?” Row 1 (“the”) is the first word, so it can only see itself its weight is forced to 1.0. Row 5 (“mat”) is last, so it sees everything. The allowed region is a lower triangle, and each row’s surviving weights still sum to 1 after softmax.

Step 3: The Transformer Block

Attention is only half of each layer. A full transformer block wraps attention and a feed-forward network together with two features that make deep stacks trainable:

• A feed-forward network (MLP) – a small two-layer network applied to each token independently, usually expanding to ~4× the model width and back. This is where a lot of the model’s “knowledge” is stored.
• Residual connections – each sub-layer adds its output back to its input, giving gradients a clean path through dozens of layers.
• Normalization (LayerNorm or the increasingly common RMSNorm), typically applied before each sub-layer (“pre-norm”) for training stability.

import torch.nn as nn

class TransformerBlock(nn.Module):
    def __init__(self, d_model, n_heads):
        super().__init__()
        self.attn  = nn.MultiheadAttention(d_model, n_heads, batch_first=True)
        self.norm1 = nn.LayerNorm(d_model)
        self.norm2 = nn.LayerNorm(d_model)
        self.mlp   = nn.Sequential(
            nn.Linear(d_model, 4 * d_model),
            nn.GELU(),
            nn.Linear(4 * d_model, d_model),
        )

    def forward(self, x, causal_mask):
        # Pre-norm attention + residual
        h = self.norm1(x)
        attn_out, _ = self.attn(h, h, h, attn_mask=causal_mask)
        x = x + attn_out
        # Pre-norm feed-forward + residual
        h = self.norm2(x)
        x = x + self.mlp(h)
        return x

A real model just stacks this block N times 32, 80, sometimes more than 100 layers with the output of one feeding the input of the next. Model size (the “7B” or “70B” you see in names) is roughly the total parameters across all these blocks plus the embeddings.

After the final block, a last linear layer projects each position back to the size of the vocabulary. A softmax turns those numbers into probabilities, and the model has its prediction for the next token.

Step 4: Generation Is a Loop

To generate text, the model predicts a probability distribution for the next token, picks one (greedily, or by sampling with a temperature setting to control randomness), appends it to the input, and runs the whole thing again. This is why LLMs are called autoregressive: each new token is conditioned on everything generated so far.

A key optimization here is the KV cache. Because past tokens don’t change, the model caches their key and value vectors instead of recomputing them every step. This is why the first token of a response can feel slow (processing your whole prompt) while subsequent tokens stream quickly.

Not All Transformers Are the Same

Decoder-only is dominant today, but it’s one of three classic arrangements. Knowing the difference explains why BERT and GPT feel like different tools.

Which Architecture Should You Care About?

For most people building on LLMs today, the answer is decoder-only that’s what “LLM” almost always means now. But if you’re choosing a model type for a specific job, walk through these questions:

• Do you need to generate free-form text (chat, code, drafts)? Decoder-only. This covers the vast majority of modern use cases.
• Do you need to turn text into a fixed vector for search or classification? An encoder-only model (or a dedicated embedding model) is cheaper and often better than asking a generative LLM.
• Is your task a clean input to output transform like translation? Encoder-decoder can shine, though large decoder-only models now handle these tasks well via prompting.

Rule of thumb:

• Generating?  Decoder-only.
• Understanding or retrieving?  Encoder-only / embedding model.
• Faithful sequence-to-sequence transformation?  Encoder-decoder.
• Not sure? Default to a decoder-only model; it’s the most general.

The post LLM Architecture Explained: What’s Actually Inside a Large Language Model appeared first on NoloWiz.

In this article we’ll clear up the confusion, explain what each service actually does, and give you a practical framework for deciding what to use for your own workloads.

The Short Answer

EC2 (Elastic Compute Cloud) gives you virtual servers in the cloud. You get an operating system, CPU, memory, and storage, and you’re responsible for everything you run on it.

ECS (Elastic Container Service) is a container orchestration service. It decides where and how your Docker containers run, restarts them when they crash, and scales them up and down.

The key thing most comparison articles bury: ECS is not an alternative to EC2 ECS runs on top of compute, and one of its two options for that compute is EC2 itself. The real decision isn’t usually “ECS or EC2.” It’s “should I manage my containers directly on EC2 by hand, or let ECS orchestrate them for me?”

What Is EC2?

EC2 is AWS’s Infrastructure-as-a-Service (IaaS) offering. When you launch an EC2 instance, you’re renting a virtual machine. You choose:

• An instance type (how much CPU and RAM – e.g. t3.micro, m5.large)
• An AMI (the base image / operating system, like Amazon Linux or Ubuntu)
• Storage (EBS volumes)
• Networking (VPC, security groups, public/private IPs)

Once it boots, that server is yours to manage. You SSH in, install software, deploy your application, patch the OS, and handle scaling. That flexibility is EC2’s biggest strength and its biggest burden you can run anything, but you also maintain everything.

EC2 is a good fit when you need:

• Full control over the operating system and environment
• Traditional (non-containerized) applications
• Specialized software that expects a full server
• Long-running workloads where you want to fine-tune the machine

What Is ECS?

ECS is a container orchestration platform. If your application is packaged as Docker containers, ECS handles the operational hard parts for you:

• Scheduling – placing containers onto available compute
• Health checks and self-healing – restarting failed containers automatically
• Scaling – running more or fewer copies based on load
• Service discovery and load balancing – integrating with an Application Load Balancer
• Rolling deployments – updating your app without downtime

In ECS you define a task definition (a blueprint describing your container image, CPU/memory, ports, and environment variables), then run it as a task or a long-running service.

Here’s a simplified task definition to make it concrete:

{
  "family": "my-web-app",
  "networkMode": "awsvpc",
  "containerDefinitions": [
    {
      "name": "web",
      "image": "123456789.dkr.ecr.us-east-1.amazonaws.com/my-web-app:latest",
      "cpu": 256,
      "memory": 512,
      "portMappings": [
        { "containerPort": 80, "protocol": "tcp" }
      ]
    }
  ],
  "requiresCompatibilities": ["FARGATE"],
  "cpu": "256",
  "memory": "512"
}

Notice there’s nothing in there about which server this runs on. That’s the point of orchestration you describe what you want running, and ECS figures out where.

The Part That Confuses Everyone: ECS Launch Types

ECS still needs actual compute to run your containers on. It offers two ways to provide it, and this is where EC2 re-enters the picture.

1.ECS on EC2 (the EC2 launch type)

You run a cluster of EC2 instances that you own and manage, and ECS packs your containers onto them. You’re still responsible for the instances patching, scaling the cluster, right-sizing but ECS handles the container-level orchestration on top.

Choose this when you want lower cost at scale, need specific instance types (GPUs, high memory), or want fine-grained control over the underlying hosts.

2.ECS on Fargate (the Fargate launch type)

Fargate is serverless compute for containers. There are no EC2 instances for you to see or manage you just specify how much CPU and memory each task needs, and AWS provisions the compute invisibly. You pay only for the resources your tasks use while they run.

Choose this when you want zero server management, variable or bursty workloads, and the simplest possible operations, and you’re willing to pay a bit more per unit of compute for that convenience.

So Which One Should You Use?

Instead of picking “ECS vs EC2” as rivals, walk through these questions:

Is your application containerized (Docker)? If no, and you don’t plan to containerize it, plain EC2 is your natural home. Run it like a traditional server.

If yes, you almost certainly want an orchestrator, which means ECS (or EKS if you specifically need Kubernetes).

If you’re using ECS, how much do you want to manage the servers?

• Want AWS to handle the compute entirely and value simplicity – ECS on Fargate
• Want lower cost at scale or need special instance types, and are comfortable managing a cluster – ECS on EC2

A quick rule of thumb:

• Single traditional app, or you need OS-level control – EC2
• Containers, and you want the least operational work – ECS + Fargate
• Containers at large scale where cost matters – ECS + EC2

A Note on Pricing

Pricing is where the trade-offs become tangible, and AWS pricing changes over time, so always confirm current rates in the AWS pricing calculator. In general terms:

• EC2 – you pay for the instance whether it’s busy or idle. Reserved Instances and Savings Plans cut costs significantly for steady workloads.
• ECS itself – there’s no extra charge for the ECS control plane. You only pay for the underlying compute.
• ECS on EC2 – you pay standard EC2 prices, so you can hit lower per-unit costs at scale but you pay for idle capacity.
• ECS on Fargate – you pay per vCPU and GB of memory that your tasks consume, billed by the second. Simpler and often cheaper for spiky or low-utilization workloads, but usually pricier per unit for constant, high-utilization ones.

Frequently Asked Questions

Is ECS a replacement for EC2? No. ECS orchestrates containers and can even run on EC2. They solve different problems and frequently work together.

Can I run containers on EC2 without ECS? Yes, you can install Docker on an EC2 instance and run containers manually. But you’d be hand-building the scheduling, health checks, and scaling that ECS gives you for free.

What about EKS? EKS is AWS’s managed Kubernetes service. It’s an alternative to ECS for orchestration (not to EC2), and it also runs on EC2 or Fargate. Choose EKS if you specifically want Kubernetes; choose ECS for a simpler, more AWS-native experience.

What’s the difference between Fargate and EC2 for ECS? Both provide the compute ECS runs on. With EC2 you manage the servers; with Fargate you don’t manage any servers at all.

Conclusion

“ECS vs EC2” is really a comparison between a building block (EC2, raw virtual servers) and a management layer (ECS, container orchestration) and ECS often sits directly on top of EC2. If you’re running traditional applications and want full control, EC2 is your foundation. If you’re running Docker containers and want automated scheduling, scaling, and healing, ECS is the tool, and your next choice is simply whether to back it with EC2 (more control, cheaper at scale) or Fargate (zero server management). Get that mental model right and the “versus” mostly disappears you’ll know exactly which piece you need, and when to combine them.

The post ECS vs EC2: What’s the Difference and Which One Should You Use? appeared first on NoloWiz.

China’s Moonshot AI Releases Kimi K3

The biggest story of the week. Beijing-based Moonshot AI (backed by Alibaba and Tencent) launched Kimi K3, a 2.8-trillion-parameter multimodal model with a 1-million-token context window and native vision. It’s positioned as the world’s largest open-weight model, with full weights scheduled for release on July 27.

• Benchmarks: Ranked #1 on Arena.ai’s Frontend Code Arena, beating Claude Fable 5 and GPT-5.6 Sol. Second overall behind Fable 5 Max on real-world task benchmarks (Artificial Analysis). State-of-the-art on
  BrowseComp (91.2/100).
• Architecture: Uses Kimi Delta Attention (hybrid linear attention) and Attention Residuals. 16 of 896 experts active per token.
• Pricing: $0.30/M input tokens (cached), $3/M uncached, $15/M output significantly cheaper than U.S. rivals.
• Agentic demo: Designed a functional 4 mm² chip in 48 hours of autonomous operation using open-source EDA tools.

OpenAI Launches GPT-Red

OpenAI has unveiled GPT-Red, an automated AI red-teaming model designed to stress-test its own systems before public release. Specializing in prompt injection attacks where malicious instructions trick AI models into exposing data or performing unauthorized actions GPT-Red uses self-play reinforcement learning to continuously discover new vulnerabilities and harden production models like GPT-5.6 Sol. The initiative arrives amid rising concerns over the security of frontier AI systems, with the Trump administration recently introducing a voluntary quarantine policy for model evaluations. While OpenAI and competitors like Anthropic are increasingly automating safety testing to build trust, industry experts warn that enterprises shouldn’t rely solely on vendor red-teaming and should still conduct their own due diligence to ensure AI tools align with their specific security and business workflows.

Thinking Machines Lab Releases Inkling

Thinking Machines released Inkling, a 975-billion-parameter Mixture-of-Experts model (41B active parameters) with a 1M-token context window, native multimodal support (text, image, audio), and controllable thinking effort for balancing cost vs. performance.

• Available for fine-tuning on the Tinker platform.
• A smaller variant, Inkling-Small (12B active), also previewed.
• Trained on NVIDIA GB300 NVL72 systems with 30M+ RL rollouts

xAI Launches Grok 4.5 into Private Beta

Elon Musk announced that xAI has officially advanced its frontier capabilities by pushing Grok 4.5 into private beta testing within SpaceX and Tesla ecosystems. Built on a massive 1.5-trillion-parameter foundation architecture, this new iteration is three times larger than the model handling production traffic on the X platform, and represents a 50% jump in scale from Grok 4.4, which rolled out only a month prior. Early internal benchmarks indicate that the model is rapidly closing the performance gap with the industry’s topmost frontier models.

Qwen3.8 is Coming: 2.4T Parameters

Alibaba has announced the upcoming open-weight launch of Qwen3.8, a massive 2.4-trillion-parameter AI model. Positioned as one of the most powerful models currently available second only to Fable 5 it is designed to compete with leading frontier AI models. While the full open-weight release is coming soon, users can immediately test the model via the Qwen3.8-Max-Preview, which has just debuted on Alibaba’s Token Plan, Qoder, and QoderWork platforms.

UN Deploys AI to Accelerate Global Methane Mitigation

The UN Environment Programme (UNEP) published a groundbreaking report detailing how artificial intelligence is being successfully leveraged to combat climate change. Utilizing the AI-driven Methane Alert and Response System (MARS), the UN has successfully processed vast feeds from over 30 satellites, enabling analysts to identify major industrial gas leaks 12 to 15 times faster than humanly possible. Crucially, the UN highlighted that the AI models were designed to be lightweight and energy-efficient, proving that advanced machine learning can drive environmental action without imposing a massive carbon footprint of its own.

The post Top AI News of the Week (July 12-July 19, 2026) appeared first on NoloWiz.

OpenAI Launches GPT-5.6

OpenAI released its GPT-5.6 family three models: Sol (flagship), Terra (value), and Luna (budget). Key highlights:

• Sol scores 80 on the Artificial Analysis Coding Agent Index (beating Claude Fable 5 by 2.8 points), uses 54% fewer tokens, and costs ~1/3 less.
• ChatGPT Work launched alongside a unified desktop interface for chat, coding, and agent tasks.
• GPT-Live full-duplex voice model launched July 8 listens and speaks simultaneously with real-time translation.
• Sol autonomously post-trained the smaller Luna model a milestone in recursive self-improvement (RSI)

Apple Sues OpenAI for Trade Secret Theft

Apple filed a federal lawsuit alleging OpenAI orchestrated a coordinated effort to extract confidential technology through 400+ former Apple employees now working at OpenAI. Two former Apple engineers are named. The suit comes weeks before OpenAI’s planned IPO, adding significant legal overhang.

Meta Releases Muse Spark 1.1

Meta launched Muse Spark 1.1, purpose-built for agentic tasks. Mark Zuckerberg posted on X for the first time in three years to announce it. Key stats:

• 54.7% on JobBench (beating Claude Opus 4.8)
• 1-million-token context window
• Supports multi-agent delegation and computer-use execution

Humanoid Robots Successfully Perform First Ever Live Surgery

A team from the University of California, San Diego (UC San Diego) has achieved a global first by performing live laparoscopic gallbladder removals on a pig using two off-the-shelf Unitree G1 humanoid robots. While the procedure was fully teleoperated by human surgeons and the robots were tethered for safety, the experiment demonstrates that affordable, general-purpose humanoids can handle complex surgical tasks on living tissue. This milestone offers a promising alternative to expensive, fixed robotic systems like the da Vinci, with researchers envisioning a future where low-cost humanoid robots extend critical surgical care to remote “medical deserts” and extreme environments like space or Antarctica.

NVIDIA & Hugging Face Open Humanoid Robotics

NVIDIA and Hugging Face announced a major robotics partnership, integrating NVIDIA’s Isaac GR00T 1.7 vision-language-action model into Hugging Face’s open-source LeRobot library. Creates a unified pipeline: teleoperate -> train -> simulate -> deploy.

BAAI’s Orca World Model

China’s Beijing Academy of AI released Orca, a world foundation model that matches specialized robotics systems trained without a single action label. Uses “unconscious learning” from unlabeled videos plus “conscious learning” from described actions. Could help solve robotics’ chronic data shortage.

Introducing BAAI WuJie·RoboBrain Orca — an early step toward Multimodal Latent World Models.
Instead of predicting only the next-token, next-frame, or next-action prediction, Orca learns world latent representations from multimodal data, and models how world state… pic.twitter.com/aV51o6mRMi

— BAAI (@BAAIBeijing) July 9, 2026

Anthropic’s Jacobian Lens

Anthropic researchers have discovered a specific internal structure in their AI model, Claude, that functions similarly to human “conscious thought.” They call this internal workspace J-space (named after the mathematical technique used to find it, the Jacobian Lens a.k.a J lens).

Think of J space like this:

• Most of Claude’s neural network works automatically.
• But when Claude needs to plan, reason, solve a puzzle, detect errors, or think about concepts, a small internal workspace becomes active.
• This workspace was not programmed by Anthropic it emerged naturally during training.

This is one of the most significant interpretability papers published recently because it moves beyond treating LLMs as opaque systems. Anthropic presents evidence for an emergent internal workspace that appears central to higher-level reasoning and can be partially observed and manipulated. While it does not imply consciousness, it provides a promising way to study how modern language models reason internally and how their behavior might be monitored for safety.

The post Top AI News of the Week (July 5-July 12, 2026) appeared first on NoloWiz.

Fable 5 Global Redeployment

Anthropic to suspend Fable 5 and Mythos 5 worldwide on June 12 after Amazon researchers found a jailbreak that let the model help identify and exploit a software vulnerability. Commerce lifted the controls on June 30, and Anthropic restored global access on July 1 across Claude.ai, the Claude Platform API, Claude Code, and Claude Cowork, with cloud-provider access (AWS, Google Cloud, Microsoft Foundry) being re-enabled “as quickly as possible.” Notably, Anthropic’s own testing later showed Opus 4.8, GPT-5.5, and Kimi K2.7 could all reproduce the same exploit meaning Fable 5 had no unique offensive capability the ban was meant to contain.

Anthropic Claude Sonnet 5 Model Launch

Claude Sonnet 5 launched on June 30 and became the default model for every Free and Pro user starting July 1 Anthropic’s biggest mass-market model launch of the year, described as the most agentic Sonnet yet, performing close to flagship Opus 4.8 on many tasks, with a 63.2% agentic coding benchmark score (vs. 58.1% for Sonnet 4.6).

Google Unveils Nano Banana 2 Lite and Gemini Omni Flash

Google DeepMind introduced Nano Banana 2 Lite and Gemini Omni Flash, two new AI media models for fast image and video generation. Nano Banana 2 Lite delivers low-cost image generation in about 4 seconds, while Gemini Omni Flash enables high-quality video creation and conversational editing. Available across Google AI Studio, the Gemini API, Enterprise Agent Platform, and consumer apps, the models can be combined to quickly turn AI-generated images into animated videos, with SynthID watermarking included for transparency.

Claude Science Beta Released

Claude Science is an AI workbench for scientists that integrates commonly used tools and packages, produces auditable artifacts, and provides flexible compute access.

Claude Science is a beta AI platform for macOS and Linux that combines 60+ life science skills with specialized agents to support genomics, proteomics, structural biology, and drug discovery. It can analyze biological data, run compute jobs on local or GPU clusters, integrate custom models and pipelines, and verify citations and calculations. Researchers are already using it to accelerate drug discovery, scientific reviews, and biomedical studies, with select projects eligible for up to $30,000 in Modal compute grants.

AI Boom Drives Massive Power Infrastructure Investments

The rapid growth of AI data centers continues to transform the energy sector. Financial analysts reported record mergers and acquisitions in utilities as companies race to build the electrical infrastructure needed for AI workloads.

• AI is now influencing entire industries beyond software.
• Reliable electricity is becoming a strategic competitive advantage.

United Nations Launches “AI for Good” Global Commission

The United Nations and the International Telecommunication Union (ITU) announced a new AI for Good Global Commission that will bring together AI company leaders, governments, and researchers to coordinate international AI governance. The inaugural meetings are scheduled for early July in Geneva.

Why it is important

• Global AI regulation is becoming more coordinated.
• May influence future AI safety standards.
• Demonstrates increasing international cooperation.

Ford Rehires Human Engineers After AI Fails to Match Quality Checks

AI can be incredibly beneficial but over reliance on it can create more problems than it solves. Ford has rehired more than 300 veteran quality inspectors after its AI-driven systems fell short of expectations. Charles Poon, vice president of vehicle hardware engineering, told reporters that “artificial intelligence is a fantastic tool, but it’s only as good as the information you use to train it” pointing out that the company had failed to preserve decades of institutional knowledge before rolling out 900 AI-powered cameras across its plants. While Ford’s chief operating officer had touted AI deployment “across the entire industrial system,”

Takeaway: AI is a powerful force multiplier, but it cannot replace decades of institutional knowledge; successful industrial automation requires a “human-in-the-loop” strategy to prevent critical quality slips.

Google Limits Gemini Capacity for Meta

Reports indicated that Google has limited Meta’s access to Gemini AI models because demand for compute resources has exceeded available capacity.

• AI compute remains one of the industry’s biggest bottlenecks.
• Demonstrates how GPU shortages are affecting even major technology companies.

Takeaway: GPU scarcity remains the industry’s ultimate bottleneck, forcing even the largest tech giants to prioritize internal resources over external partnerships.

The post Top AI News of the Week (June 28-July 5, 2026) appeared first on NoloWiz.

OpenAI unveils its first custom chip, “Jalapeño,” with Broadcom

On June 24, OpenAI and Broadcom revealed Jalapeño, OpenAI’s first custom-designed AI chip, with engineering samples physically delivered to Sam Altman and Greg Brockman by Broadcom CEO Hock Tan. The chip targets roughly 50% cheaper LLM serving, with prototype data center deployment expected by end of 2026 and production ramping in 2027-2028.

Anthropic accuses Alibaba of a massive distillation attack

A letter Anthropic sent to US Senators Tim Scott and Elizabeth Warren became public this week. Anthropic accused Alibaba and its Qwen AI lab of running “the largest known distillation attack on Anthropic to date,” alleging attackers used 25,000 fraudulent accounts over six weeks to run 28.8 million exchanges between April 22 and June 5, 2026, seeking to extract agentic reasoning, software engineering proficiency, and long-horizon task completion capabilities.

Google loses more top AI researchers to Anthropic and OpenAI

Two leading Gemini researchers, Jonas Adler and Alexander Pritzel, are planning to leave Google for Anthropic, adding to a wave of exits. The company had already lost Nobel laureate John Jumper to Anthropic and star researcher Noam Shazeer to OpenAI, with the departures rattling investors and pulling Alphabet’s stock down.

Google delays Gemini 3.5 Pro

Google quietly pushed the general availability of Gemini 3.5 Pro from June to July 2026, with testers flagging issues around token efficiency and long-horizon task performance.

The industry shifts from “tokenmaxxing” to efficiency

The era of spending freely on AI is cooling off. As OpenAI and Anthropic head toward historic IPOs both filed confidentially in early June at valuations nearing $1 trillion enterprise customers are growing reluctant to pour money into frontier models without clear returns. The trend has a name shift behind it: from “tokenmaxxing” (chasing maximum capability at any cost) to prioritizing efficiency. The clearest example came from Lindy CEO Flo Crivello, who moved 100% of his company’s traffic off Anthropic’s Claude to the cheaper Chinese open-weight model DeepSeek, watching his costs “crash to the ground” and calling it a matter of business survival. With Anthropic and OpenAI slower to cut token prices lately, and Microsoft, Amazon, and Google all pushing low-cost alternatives, the pressure on premium AI pricing is only mounting.

Mythos 5 Partially Restored – The Anthropic Saga Continues

Two weeks after the Commerce Department abruptly banned Anthropic’s Mythos 5 and Fable 5 models, the Trump admin partially lifted restrictions on Mythos 5, allowing it to be redeployed to critical infrastructure operators and cyber defenders. Fable 5 remains banned. Commerce Secretary Howard Lutnick told Anthropic its efforts “yielded significant progress,” and Anthropic agreed to future model review protocols. The Pentagon had previously designated Anthropic a national security risk over ethical concerns about AI in warfare.

OpenAI’s GPT-5.6 Release

The big release was the GPT-5.6 family, announced June 26. It’s a set of three models on a new naming structure where the number (5.6) is the generation, and Sol/Terra/Luna are durable tiers that can be upgraded independently over time — essentially a “good/better/best” lineup.

• Sol (Flagship): Designed for deep reasoning, coding, and biology. It features a massive ~1.5M token context window and two new modes:
  • Max Mode: Allows the model to “think” longer before answering.
  • Ultra Mode: Spins up multiple AI agents to solve complex, multi-step problems.
• Terra: The balanced option, offering performance close to GPT-5.5 but at roughly half the cost.
• Luna: The smallest and fastest model, optimized for quick responses and high-volume, low-complexity tasks.

The post Top AI News of the Week (June 21-28, 2026) appeared first on NoloWiz.

1. Stash your changes.
2. Checkout main.
3. Fix the bug.
4. Commit.
5. Checkout back to your feature branch.
6. Stash pop. Sound tedious? You’re not alone.

But there’s a better way. Today, I’m going to introduce you to one of Git’s most underrated features: git worktree!

What Is Git Worktree?

In simple terms, git worktree lets you work on multiple branches of the same repository simultaneously, each in its own directory.

Normally, Git only lets you be on one branch at a time. To switch branches, you either commit, stash, or discard your current work. With worktrees, you get separate working directories that all share the
same .git folder meaning you can be on feature-x in one window and main in another, without any stashing or switching.

The Problem It Solves

Imagine you are working on a feature branch (feature-x) and a critical bug appears on the main branch.

• Without worktree: You stash your changes, checkout main, fix the bug, commit, and then stash pop back to your feature.
• With worktree: You simply open a second terminal or directory for main, fix the bug, and leave your feature branch untouched in the first directory.

That’s it. No stashing. No context switching. No lost mental momentum.

How to Use It

1. Create a new worktree

Run the following command :

git worktree add -b fix-bug ../fix-bug main

This creates a new folder (../fix-bug), creates a new branch named fix-bug based on main, and checks it out for you automatically.

2. List All Worktrees

Run the below command to list all worktrees :

git worktree list

Output :

/path/to/repo       (master)
/path/to/fix-bug    (fix-bug)

3. Work on the new branch

Navigate into your new directory (cd ../fix-bug). Here, you can make commits, run tests, or mess with dependencies completely independently from your original repository. Your main repo’s working directory remains completely untouched.

4. Remove a Worktree

When you are finished with your changes and have pushed or merged your work, you can clean up:

git worktree remove ../fix-bug

Note: This safely deletes the ../fix-bug directory. It does not delete the branch itself from Git history. If you want to completely delete the branch too, you can now safely run git branch -d fix-bug from your main repository directory.

Common Use Cases

• Fixing urgent bugs – Work on main for a hotfix while keeping your feature branch clean in the main terminal.
• Testing – Test a specific branch without having to switch your current working state.
• Code reviews – Pull down a colleague’s PR in a separate folder without merging it into your local repo.
• Documentation – Work on both code and docs in parallel without switching branches.
• AI Agents & Copilots – In 2026, we see a massive spike in worktree usage driven by coding AI agents (like Claude Code, GitHub Copilot Workspace, or OpenAI Codex). These tools often spin up isolated background git worktrees to write features, run automated tests, and fix bugs without disrupting the developer’s active screen.

Git Worktree Anti-Patterns to avoid

1. Checking Out the Same Branch Twice

git worktree add ../fix main   # fails if already on main

Why: Git hard-blocks this. Each worktree must be on a unique branch.

2. Rebasing/Amending a Branch Another Worktree Is Using

# Worktree A is on feature-x
# In main worktree — DON'T do this:
git rebase -i HEAD~3 feature-x   # corrupts worktree A's HEAD

Why: History rewrite orphans the other worktree’s commits.

3. Putting Worktrees Inside the Repo Folder

git worktree add ./fix-bug -b fix-bug main   # inside repo

Why: Causes .gitignore headaches, accidental commits of worktree folders, and confusing git status output. Always place worktrees outside or as siblings:

git worktree add ../fix-bug -b fix-bug main  # Correct - sibling folder

4. Forgetting to Remove Stale Worktrees

# Manually deleted the folder but never ran:
git worktree remove fix-bug

Why: Git still tracks it internally. Builds up ghost entries. Always clean up:

git worktree prune    # removes stale entries
git worktree list     # verify

5. Running Git Commands That Affect Shared State From Wrong Worktree

# From fix-bug worktree — dangerous:
git branch -D feature-x     # deletes branch another worktree is on
git tag -f v1.0              # moves a tag others depend on
git gc --aggressive          # can corrupt shared object store mid-work

Why: All worktrees share the same .git – destructive commands affect everyone.

6. Using Worktrees With Submodules Carelessly

git worktree add ../feature -b feature main
# Submodules are NOT automatically initialized in new worktree

Why: Submodule state is not automatically carried over. You must manually run:

cd ../feature
git submodule update --init

Final Thoughts

The git worktree is one of those features that, once you start using it, you wonder how you ever lived without it. It’s lightweight, it’s built into Git (no plugins needed), and it saves you from the stashing dance every time.

The post Mastering Git Worktree: How to Work on Multiple Branches at Once appeared first on NoloWiz.

What about India – where 1.4 billion people work across farming, construction, IT, banking, healthcare, and government? Where does AI actually threaten livelihoods, and where is the workforce relatively safe?

I wanted to see this visually. So I built it. Inspired by Andrej Karpathy’s jobs project which analyzed 342 US occupations from BLS data – I built the Indian version using NCS Portal data. You can visit here .

What the map shows

The visualization is an interactive treemap of the Indian job market, covering 10 major sectors and ~500 occupations from the National Career Service portal , India’s official government career database based on NCO-2015 classification.

Each rectangle is one occupation. Two visual signals:

• Size – how many people work in that occupation. A farmer’s rectangle is enormous. A software architect’s is tiny.
• Color – how exposed that occupation is to AI disruption, scored 0–10. Green means relatively safe. Red means high risk.

The picture that emerges is striking, and very different from the American version of this story.

What India’s map actually looks like

Agriculture dominates. Nearly 40% of India’s workforce is in farming – cultivators, dairy workers, livestock farmers. These jobs score 2–3 out of 10 on AI exposure. They require physical presence, seasonal judgment, and local knowledge that AI cannot replicate at scale, especially in India’s fragmented smallholder farming context. The agriculture block is a massive sea of green.

Construction is the second green giant. With 60+ million workers, construction scores 3/10. Masons, welders, electricians, plumbers – physical skills in unpredictable environments that robots still can’t handle reliably. India’s infrastructure boom under Smart Cities and PM Gati Shakti is creating more of these relatively safe jobs.

IT-ITeS is a small red island. India’s famous software and BPO sector employs far fewer people than agriculture – but scores 7–9 out of 10 on AI exposure. Software developers, data analysts, business process managers, content writers – these are exactly the jobs that LLMs are already eating into. The IT block is tiny but blazing red.

BFSI tells a split story. Bank tellers and loan document processors are highly exposed (7–8/10). Relationship managers and wealth advisors are more protected (5/10) because trust still drives financial decisions in India. The branch banking model that employs hundreds of thousands is under significant pressure.

Telecom is surprisingly high risk. Customer care agents, billing processors, network documentation staff – a huge portion of India’s telecom workforce is in roles scoring 7–8/10. Jio disrupted pricing; AI is disrupting the workforce. Tower technicians and field engineers score much lower (3–4/10) because their work is physical.

Logistics is a tale of two workforces. Delivery workers and warehouse staff score low (2–3/10) – physical work, last-mile human judgment. But dispatch coordinators, route planners, and logistics analysts score 6–7/10. Zomato and Swiggy’s gig economy has created millions of AI-safe delivery jobs while quietly automating the planning layer above them.

Organised Retail is in transition. Cashiers and billing staff score high (7/10) – self-checkout and UPI are already replacing them. But floor staff, visual merchandisers, and store managers score moderate (4–5/10). The kirana store owner scores low – hyperlocal relationships and informal credit systems are hard for AI to replicate.

Public Administration scores higher than people expect. Data entry clerks, document processing officers, and administrative assistants in government score 6–7/10. The institutional inertia of Indian bureaucracy will delay automation, but it won’t prevent it. Millions of aspirational government job seekers are training for roles that AI will significantly reshape within a decade.

Healthcare is the interesting middle ground. Doctors and specialists score moderately (5–6/10) because diagnosis and patient relationships still require human judgment. But medical transcriptionists, billing clerks, and hospital administrative staff score 8–9/10. AI will hollow out healthcare administration long before it touches clinical care.

Education sits in the amber zone. With 10 million+ teachers in India, education scores 4–5/10. The human relationship at the core of teaching is protective – but administrative staff, content creators, and exam evaluators score much higher.

The full picture across 10 sectors

The uncomfortable takeaway

India has spent 30 years building an economy on the back of knowledge work IT services, BPO, back-office processing that is precisely what AI automates best. Meanwhile, the jobs employing most Indians farming, construction, delivery are safe not because they’re valuable but because they’re physical and informal.

Construction workers building smart cities. Delivery workers powering e-commerce. Farmers feeding a billion people. All relatively safe from AI.

Software engineers. Bank clerks. Government data entry operators. Call centre agents. All highly exposed.

The map doesn’t have answers. But it makes the question visible.

The post I Built a Map of Which Indian Jobs Are Most at Risk from AI appeared first on NoloWiz.

Docker Model Runner (DMR)

Docker Model Runner (DMR) is a tool built into Docker Desktop and Docker Engine that makes it easy to pull, run, and serve AI/LLM models locally directly from Docker Hub, any OCI-compliant registry, or Hugging Face. Models can be pulled from model resgistry and stored locally.

DMR has the following key features :

• Serves models via OpenAI and Ollama-compatible APIs, so existing apps can plug right in Docker
• Models load into memory only at runtime and unload when not in use to save resources Docker
• Following inference engines are supported
  • llama.cpp (default, all platforms)
  • vLLM (high throughput, NVIDIA)
• Image generation via diffusers
• Integrates with AI coding tools like Cline, Continue, Cursor, and Aider Docker
• Works with Docker Compose and Testcontainers

Step 1: Enable Docker Model Runner

First we need to install docker desktop/ docker by following the getting started docker guide. To enable docker model runner do the following.

Docker Desktop: Go to Settings -> AI tab, then enable Docker Model Runner. Optionally enable GPU-backed inference if you have a supported NVIDIA GPU.

Docker Engine (Linux): Install the plugin, then test it:

sudo apt-get update
sudo apt-get install docker-model-plugin

Now we can verify docker model command by running the below command

docker model version

Step 2: Pull a Model

Next we need to pull a model from Docker Hub.

docker model pull ai/smollm2:360M-Q4_K_M

Or pull directly from HuggingFace:

docker model pull hf.co/bartowski/Llama-3.2-1B-Instruct-GGUF

Models are cached locally after the first pull.

Docker models can be installed using docker desktop as shown below

Step 3: Run the Model

Run the below command to run the model with interactive CLI :

docker model run ai/smollm2:360M-Q4_K_M

How to use Model API

By default, Docker Model Runner may only be accessible via a Unix socket or internal Docker networking. To call it from your host machine (e.g., via curl or Postman), you must explicitly enable TCP host access. As we have enabled this in the docker desktop we can use the API.

For docker CLI use the below command enable it

docker desktop enable model-runner --tcp=12434

Docker Model Runner uses an OpenAI-compatible API, but the path includes the engine and model name. Base URL structure:

• From Host: http://localhost:12434/v1
• From inside a Container: http://model-runner.docker.internal:12434/v1

Testing the API with Postman.

We can connect with OpenAI comptible libraries, here is an example of Python code :

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:12434/engines/v1",
    api_key="not-needed",
)

response = client.chat.completions.create(
    model="ai/smollm2:360M-Q4_K_M", messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)

Context Size

The context size is the total token budget for each request, split between what you send in and what the model generates back. As per DMR documentation default context size for the engines :

• llama.cpp – 4096
• vLLM – Uses the model’s maximum trained context size

We can configure the model context size using the below command :

docker model configure --context-size 8192 ai/qwen2.5-coder

When to Use Docker Model Runner

Use docker model runner in the following scenarios.

• Local development & testing – Local development without costly API calls, or privacy concerns from cloud APIs.
• Privacy-sensitive workloads – To keep confidential data fully under your control.
• Docker-native workflows – Use familiar docker model pull/run commands with no new toolchain to learn.
• Multi-container AI apps with Compose – Define models directly in compose.yml alongside your app services with zero extra glue code.
• Offline / edge environments – Run models locally where cloud API access isn’t reliable or allowed.
• CI/CD pipelines – Pull, tag, version, and deploy models like any other artifact, no GPU cluster required

Quick Troubleshooting 

1. To check whether Docker Model Runner (DMR) run the below command :

docker model status

2. To list pulled models use the below command :

docker model ls

3. To display detailed information about a specific model

docker model inspect <model_name>

E.g docker model inspect ai/smollm2:360M-Q4_K_M

4. Test basic connectivity (List Models):

curl http://localhost:12434/v1/models

Conclusion

Docker model runner makes it easier to run AI models locally without much problem and staying with the docker ecosystem. Run Large Language Models Locally with Simple APIs.

The post Run AI Models with Docker Model Runner: A Step-by-Step Guide appeared first on NoloWiz.

Agent Skills

The open Agent Skills standard was introduced by Anthropic. Agent skill is a modular add-on that gives AI agents new abilities, from coding best practices to video editing. Skills are a new open standard for packaging reusable expertise into modular units that any compatible AI agent can discover, load, and apply on demand. Think of them as plugins for your agent’s brain: instead of repeating the same long prompt every time you want your AI to follow your team’s React conventions or generate a proper Dockerfile, you install a skill once and the agent applies it automatically whenever relevant.

Skill Folder Structure

At its core, a skill is simply a directory that contains a SKILL.md file. This file holds essential metadata such as the skill’s name and description along with detailed instructions that guide an agent in completing a specific task.

my-skill/
├── SKILL.md          # Required: instructions + metadata
├── scripts/          # Optional: executable code
├── references/       # Optional: documentation
└── assets/           # Optional: templates, resources

Agent Skills Format

A skill is a directory containing atleast one file called SKILL.md. Optional directories such as scripts/, references/, and assets/ can be added to provide extra functionality and resources for your skill.

SKILL.md file

The SKILL.md file must begin with YAML frontmatter (Frontmatter refers to the introductory section of a document or publication that contains information about the content), followed by the main content written in Markdown.

---
name: skill-name
description: A description of what this skill does and when to use it.
---

The name and description fields are necessary other optional fields includes allowed-tools, metadata,license.

• name – Lowercase letters, numbers, and hyphens only(Max 64 characters) E.g name: code-review
• description – Clear description of what the skill does and when to use it( max 1024 characters)
• license(optional) – The license applied to the skill, e.g – license: Proprietary. LICENSE.txt has complete terms
• allowed-tools(optional) – The space limited allowed tools to use e.g : allowed-tools: Read, Grep
• metadata(optional) – Additional data as key value pair
• compatibility (Optional) – Whether this skill is intended for a particular environment e.g : compatibility: Designed for Claude Code (or similar products)

Finally we have skill body it contains skill instructions. It should have following recommended sections

• Step-by-step instructions
• Examples of inputs and outputs
• Common edge cases

Here is a simple example of SKILL.md file.

---
name : Weather Retriever
description: Fetches real-time weather data and forecasts for any city globally. Use this when the user asks about current conditions or travel planning.
---

## Instructions
1. Extract the `city_name` and `units` (metric/imperial) from the user prompt.
2. If the city is missing, ask for clarification before proceeding.
3. Call the `get_weather_data` function using the extracted parameters.
4. Format the output into a friendly, 2-line summary for the user.

## Tools & Resources
- **Code:** `weather_api_client.py`
- **Data:** `city_codes.json` (for validation)

## Constraints
- Do not provide forecasts beyond 7 days.
- Always include the "Last Updated" timestamp in the response.

Optional directories

• scripts/ – contains executable code that agents can run to perform actions or computations. (Python,Javascript or bash)
• references/ – holds extra documentation and reference files that the agent can read on demand. for example REFERENCE.md for detailed reference
• assets/ – stores static resources like templates, images, or data files used by the skill

How Agent Skills Work

Agent skills has following life cycle :

• Discovery – The agent scans available skills and reads their names and descriptions to understand what capabilities are available.
• Activation – When a task matches a skill’s purpose, the agent loads and reads the full SKILL.md instructions.
• Execution – The agent follows the skill’s instructions, using any scripts, assets, or references required to complete the task.

You can refer the Skills by Anthropic on this GitHub repo.

MCP vs Agent Skills

The key differences between MCP(Model context proctocol) and agent skills are listed below.

Best Practices with Agent skills

Follow these best practices to work with agent skills :

• Create a dedicated folder per skill (e.g., pdf-parsing/) inside a skills/ directory
• Define “When to use” and “How to use” sections in SKILL.md with clear steps, parameters, and examples.
• Keep SKILL.md within 500 lines. If it goes beyond that, evaluate whether some sections should be moved into separate reference files.
• If you are using third party skills , enusre that it does not contains any malicious instructions/code.

When to use Agent Skills

Use agent skills in the following scenarios :

• When tasks are reusable (E.g Web research skill, blog outline generator skill)
• When You want separation between “Brain” and “Tools”, think like this LLM as brain(reasoning) and skills as hands(execution),If your system only needs thinking then no skill needed, if your system needs doing then skills are required
• Avoid skills for one-off tasks (use prompts) or real-time external access (use MCP/tools).

Security Risks

The following secuirty risks are associated with third party agent skills :

• Malicious Code Injection : Third-party skills often bundle executable instructions or scripts (e.g., hidden curl commands in markdown) that AI agents execute blindly, enabling data exfiltration, backdoors, or system compromise without human review
• Privilege Escalation : Skills frequently request excessive permissions—like sudo access, credential stores, or root execution far beyond stated needs, amplifying damage if exploited.

OpenClaw and VirusTotal are now collaborating to scan ClawHub, the marketplace for agent skills. You can also use Skill Scanner by Cisco for free.

skill-scanner scan <skill folder path>

Conclusion

Agent Skills are transforming how AI agents move from simple chatbots to capable task executors. By packaging instructions, tools, and structured workflows into reusable skill modules, you can build agents that are scalable, maintainable, and easier to extend.

The post Agent Skills: Complete Beginner’s Guide to AI Agent Skills and Best Practices appeared first on NoloWiz.