When submitting an HD image drawing to GPT-5 via the Azure OpenAI API, the returned bounding box coordinates are consistently off by multiple pixels. This systematic error occurs in the same direction and by the same factor every time. The model is not hallucinating; rather, it is performing coordinate arithmetic on an image it never actually processed.
This same behavior occurs across other Generative AI models, such as Gemini, Claude, and Qwen. In this blog post, we will explore why this happens and discuss potential solutions.
- What LLM views when you send a high-resolution image
- The ChatImageDetailLevel parameter and what each value actually does
- Fixing the coordinate error: pre-scale before the API call
- What changed in GPT-5.4 and GPT-5.5
- How do different models compare on resolution for this kind of workload
- Qwen3-VL: no fixed pipeline, fully configurable resolution
- Where to find the official pixel limits for each model
What LLM views when you send a high-resolution image
When you send an image using ChatImageDetailLevel.High via the Azure OpenAI C# SDK, OpenAI’s servers run a three-step pipeline before the model sees anything:
- Scale the image to fit within a 2048×2048 bounding box, preserving aspect ratio.
- Scale further so the shortest side equals 768 pixels.
- Divide into 512×512 tiles. Each tile costs 170 tokens, plus a base cost of 85 tokens.
For a 4960×3507 image (aspect ratio 1.414), step 1 produces approximately 2048×1448. Step 2 brings that down to 1086×768. That is what the model sees. The OpenAI vision documentation covers this calculation in full. The token cost for this image at high detail is 85 + (170 × 6) = 1,105 tokens. Manageable. But the coordinate space is now wrong.
This pipeline applies to every GPT model on Azure OpenAI that uses the tiled vision approach: gpt-4, gpt-4o, and early GPT-5 deployments. The detail level controls whether you hit this pipeline, not how much original resolution survives.
The ChatImageDetailLevel parameter and what each value actually does
The Azure.AI.OpenAI SDK exposes image detail via the ChatImageDetailLevel enum, passed to ChatMessageContentPart.CreateImagePart():
// Pass a URL with explicit detail level
var imagePart = ChatMessageContentPart.CreateImagePart(
new Uri("https://your-storage.blob.core.windows.net/drawings/plan.png"),
ChatImageDetailLevel.High
);
// Or pass binary data
var imagePart = ChatMessageContentPart.CreateImagePart(
BinaryData.FromBytes(imageBytes),
"image/png",
ChatImageDetailLevel.High
);
| Detail level | What the model receives | Token cost (4960×3507) |
|---|---|---|
| Low | 512×512 version of the full image | 85 tokens flat |
| High | Tiled at 512px after 768px rescale | 1,105 tokens |
| Auto | OpenAI decides; usually High for large images | Varies |
Auto is not a safe default for any workload where coordinates, spatial layout, or fine text are important. You are handing the decision to OpenAI’s preprocessing logic, and that logic is not documented in a way that guarantees consistency across deployments or model versions. Set it explicitly.
Fixing the coordinate error: pre-scale before the API call
Pre-scale the image to 1086×768 before sending it. Tell GPT the image is that size. When coordinates come back, multiply by the inverse scale factor to restore full-resolution coordinates. Nothing changes downstream.
// Step 1: Calculate the effective resolution OpenAI will use
// For a 4960x3507 image with detail:high:
// Fit in 2048x2048 -> 2048x1448 (limited by width)
// Scale shortest side to 768 -> 1086x768
const int EffectiveWidth = 1086;
const int EffectiveHeight = 768;
const double ScaleX = 4960.0 / EffectiveWidth; // 4.57x
const double ScaleY = 3507.0 / EffectiveHeight; // 4.57x
// Step 2: Pre-scale the image to match what OpenAI will produce internally
using var originalImage = Image.Load(imageBytes);
originalImage.Mutate(x => x.Resize(EffectiveWidth, EffectiveHeight));
using var ms = new MemoryStream();
await originalImage.SaveAsPngAsync(ms);
var scaledBytes = ms.ToArray();
// Step 3: Send the pre-scaled image; prompt references the actual sent size
var prompt = $"The image is {EffectiveWidth}x{EffectiveHeight}. " +
"Return bounding box coordinates as [x, y, width, height].";
var imagePart = ChatMessageContentPart.CreateImagePart(
BinaryData.FromBytes(scaledBytes),
"image/png",
ChatImageDetailLevel.High
);
// Step 4: Restore full-resolution coordinate space after the call
var fullResX = (int)(modelX * ScaleX);
var fullResY = (int)(modelY * ScaleY);
Calculate the effective resolution dynamically, not as a hardcoded constant. If the input image dimensions change, the effective resolution changes with them. The formula: fit the image in a 2048×2048 box preserving aspect ratio, then scale the shortest side to 768. Document the scale factor in code with a comment explaining why it exists, not just what it is.
What changed in GPT-5.4 and GPT-5.5
This fix was the correct approach for early GPT-5 deployments because the 768px ceiling was baked into the pipeline. GPT-5.4 changed thist by introducing a new detail level: original. It supports full-fidelity perception up to 10.24 million total pixels or a 6,000-pixel maximum dimension, whichever is lower. For a 4960×3507 image (17.4MP), there is still some downscaling involved, but the effective resolution is far higher than 1086×768.
GPT-5.5 went further: when image_detail is unset or set to auto, the model now uses original behaviour by default. OpenAI documented strong gains in localisation accuracy and click precision at these detail settings.
At the time of writing, the ChatImageDetailLevel enum in the .NET SDK does not yet expose an Original value. You can pass it as a raw string through the underlying request options, or track the openai-dotnet changelog for when it lands in a stable release. If you are already on a GPT-5.5 deployment and using Auto, you are likely benefiting from the improved handling by default. Still worth setting it explicitly in production.
How do different models compare on resolution for this kind of workload
The tiled 768px pipeline is an OpenAI architectural choice. Other providers approach this differently, and for coordinate-precision tasks the differences are significant.
| Model | Effective resolution | Where |
|---|---|---|
| GPT-5.4 | 1086×768 | Azure OpenAI / AI Foundry |
| GPT-5.5 | up to ~3840×2716 | Azure OpenAI / AI Foundry |
| Claude Sonnet 4.x | ~2659×1880 | Azure AI Foundry / Anthropic API |
| Gemini 1.5/2.0 Pro | ~3072×2172 | Vertex AI / Google AI |
| Qwen3-VL | Configurable, no fixed pipeline | Self-hosted / Hugging Face |
Claude is the most practical option if you need higher resolution and want to stay in the Azure AI Foundry ecosystem. It processes at roughly 2.4× higher effective resolution than early GPT-5 for this image, and it is available as a managed model directly in Foundry. Gemini gives better native resolution than Claude, but brings a Google Cloud dependency with it.
Qwen3-VL: no fixed pipeline, fully configurable resolution
Qwen3-VL uses a native dynamic-resolution Vision Transformer trained from scratch. There is no hardcoded 768px rescale step. Images are processed at their native aspect ratio and the number of visual tokens scales with input size. You set the resolution window via image_min_pixels and image_max_pixels in the processor config. For the 8B Instruct variant, recommended defaults are:
--image_min_pixels $((256 * 32 * 32)) # ~262K pixels minimum
--image_max_pixels $((1280 * 32 * 32)) # ~1.3M pixels maximum
For engineering drawings or any workload where fine text and precise coordinates matter, push image_max_pixels higher. The architecture handles it; the constraint is VRAM.
| Model | Size | Architecture | Hugging Face |
|---|---|---|---|
| Qwen3-VL-8B-Instruct | 8B dense | Standard | Qwen/Qwen3-VL-8B-Instruct |
| Qwen3-VL-32B-Instruct | 32B dense | Standard | Qwen/Qwen3-VL-32B-Instruct |
| Qwen3-VL-235B-A22B-Instruct | 235B MoE (22B active) | MoE | Qwen/Qwen3-VL-235B-A22B-Instruct |
| Qwen3-VL-235B-A22B-Thinking | 235B MoE + reasoning | MoE | Qwen/Qwen3-VL-235B-A22B-Thinking |
No managed API, no Azure AI Foundry integration. Self-hosted only. The 8B and 32B variants run on a single A100 or H100. The 235B MoE needs multi-GPU. If your workload has to stay in Foundry, Claude is the more practical call. Qwen3-VL is worth it when you need maximum resolution control and you are running your own inference stack.
The pre-scaling fix works with all of these models. The scale factor just changes depending on each model’s actual processing resolution. That is all it is: know what the model sees, match the coordinate space to it, multiply back at the end.
Where to find the official pixel limits for each model
None of this is hidden, but it is spread across four different documentation sites. These are the exact pages where each provider documents their image processing limits, resolution pipelines, and token costs.
OpenAI / Azure OpenAI
The tiling algorithm, detail level behaviour, and token cost formula are all documented on the OpenAI Vision guide. Look for the section titled “Calculating costs”. It includes the full step-by-step pipeline (2048 box, 768 shortest side, 512 tiles) and worked examples for different image sizes.The OpenAI vision docs: the tiling pipeline and token cost calculation. This is where the 2048/768/512 numbers come from.
Anthropic Claude
Claude’s image handling is documented on the Anthropic Vision documentation. The key numbers: maximum input is 8,000×8,000 pixels at 5MB. Effective processing resolution caps the long edge at 1,568 pixels (roughly 1.15MP for a square image). The token formula is (width × height) / 750. There are no detail modes to select; you send the image and the model handles it.Anthropic vision docs: the pixel limits table and token cost formula for Claude models.
Google Gemini
Gemini’s image resolution limits and token costs are on the Gemini API vision documentation. Gemini 1.5 and 2.0 Pro support up to 3,072 pixels on the long edge before downscaling. The token cost varies by model tier. For Vertex AI deployments, the same limits apply and are documented in the Vertex AI multimodal image understanding guide.Gemini API vision docs: maximum resolution and token cost per image across model tiers.
Qwen3-VL
Qwen3-VL does not publish a fixed resolution limit in the same way because the limit is configurable. The processor config options (image_min_pixels, image_max_pixels, and the 32×32 patch arithmetic) are documented in the model cards on Hugging Face: Qwen3-VL-8B-Instruct and Qwen3-VL-32B-Instruct. The Qwen3-VL GitHub repository has the full technical details including the DeepStack architecture and MRoPE positional embedding changes from Qwen2.5-VL.Qwen3-VL model card on Hugging Face: the processor config parameters controlling effective input resolution.
All of these limits can change with model updates. The token formulas in particular have shifted between major versions. If you are hardcoding scale factors or pixel budgets, tie them to a specific model version and add a comment pointing to the documentation page where you got the numbers.