Amazon Web Services (AWS) has announced the integration of NVIDIA’s Nemotron 3.5 Super VL model into Amazon SageMaker JumpStart, a platform designed to accelerate machine learning development. This advancement allows users to deploy a sophisticated vision-language model (VLM) capable of processing and understanding images, video, and text simultaneously to generate textual outputs.
The Chenab Times has learned that Nemotron 3.5 Super VL is a large-scale model, featuring a 120 billion-parameter hybrid Mixture-of-Experts (MoE) language backbone with 12 billion active parameters. It is complemented by a C-RADIO vision encoder and supports an extensive context length of up to one million tokens. The model also incorporates a built-in Multi-Token Prediction (MTP) head that facilitates speculative decoding, offering configurable reasoning capabilities per request.
Advanced Multimodal Capabilities and Architecture
NVIDIA engineered Super VL with specific applications in mind, including document intelligence, graphical user interface (GUI) understanding for agentic systems, video question answering, and other multimodal agentic applications. SageMaker JumpStart offers two variants of the model: NVFP4, which has been quantized from BF16 precision using distillation for optimized inference, and the BF16 full-precision reference weights for more extensive customization.
The model’s architecture comprises a language backbone with 88 layers, integrating Mamba-2 and MoE layers, alongside attention layers and a shared MTP head. This backbone is initialized from Nemotron 3 Super Base and distilled from Nemotron 3 Ultra Base. The C-RADIO vision encoder processes visual inputs, and a vision-language connector bridges them to the language backbone’s embedding space. Both the encoder and connector are maintained in BF16 precision across both variants.
The training process for Nemotron 3.5 Super VL involves several stages, including continued pre-training, long-context and software-engineering mid-training, and vision-language training. This is followed by multimodal supervised fine-tuning (SFT) on diverse datasets, reinforcement learning with human feedback, multi-teacher on-policy distillation, and final training of the MTP head. The NVFP4 checkpoint is a product of NVIDIA Model Optimizer and quantization-aware distillation, preserving key components in BF16 while operating with 4-bit weights and activations on Blackwell GPUs.
Key Use Cases and Deployment Variants
Nemotron 3.5 Super VL is particularly suited for agentic workloads requiring visual comprehension. Its capabilities extend to document intelligence, enabling optical character recognition (OCR), chart and table analysis, and processing of multi-page documents. Each rendered page image consumes approximately 1,536 tokens of context. The model can also interpret GUIs for computer-use agents and analyze short video clips for summarization and question answering.
Users can leverage the BF16 variant for extensive customization, including SFT, reinforcement learning, distillation, domain adaptation, and creating custom quantized versions. The NVFP4 variant is optimized for inference, providing efficient latency and throughput on NVIDIA RTX PRO 4500 Blackwell Server Edition GPUs. The BF16 variant is available on ml.p5en.48xlarge instances equipped with NVIDIA H200 GPUs.
SageMaker JumpStart provides two deployment configurations for the NVFP4 variant: one for optimized inference with a focus on cost-performance, and another as a full-precision reference for further fine-tuning. The BF16 variant is available as a full-precision reference model.
Performance and Accessibility
Benchmark results indicate that the NVFP4 checkpoint performs comparably to the BF16 variant across various multimodal tasks, including long-document understanding, OCR, chart reasoning, video understanding, and visual computer use. The NVFP4 variant shows a slight edge in OCRBench v2 and OSWorld-Verified benchmarks.
Inference performance tests on SageMaker JumpStart demonstrate the model’s throughput. The NVFP4 variant on ml.g7.24xlarge instances achieved approximately 63 tokens per second for output, with a time to first token of around 0.45 seconds. The BF16 variant on ml.p5en.48xlarge instances delivered a higher throughput of approximately 140 tokens per second.
Deployment on SageMaker JumpStart can be initiated through SageMaker Studio, the SageMaker Python SDK, or directly from the Hugging Face model page. Prerequisites include an AWS account with appropriate IAM roles and service quotas for the selected instance types. The deployment process typically takes about 10 minutes.
Users can interact with the deployed model via a vLLM-powered chat completions API. Media inputs, such as images and video, must be sent as base64 data URLs. The model supports reasoning, which can be enabled or disabled per request, and is particularly useful for complex analysis or direct extraction tasks, respectively. NVIDIA recommends specific temperature and top_p settings for text, image, and video inferences to optimize results.
The Chenab Times News Desk
