NVIDIA Certified Associate - Generative AI and LLMs (NCA-GENL) Fact Sheet¶

Exam Overview¶

Exam Code: NCA-GENL Exam Name: NVIDIA Certified Associate - Generative AI and LLMs Duration: 60 minutes Questions: 50-60 multiple choice questions Passing Score: 70% (scaled scoring) Cost: $125 USD Valid For: 2 years Delivery: Online proctored

📖 NVIDIA Certification Portal - Registration and official details 📖 NVIDIA Deep Learning Institute - Official training courses 📖 NVIDIA Developer Resources - Developer tools and documentation

Target Audience¶

This certification is designed for: - AI engineers building generative AI applications - ML engineers working with large language models - Data scientists exploring LLM capabilities - Software developers integrating LLMs into products - Technical professionals seeking validated AI expertise

📖 NVIDIA Training Catalog - Available DLI courses 📖 NVIDIA AI Enterprise - Enterprise AI platform overview

Domain 1: LLM Fundamentals (25%)¶

This domain covers the core architecture, training methodology, and theoretical foundations of large language models.

1.1 Transformer Architecture¶

Key Concepts: - The transformer architecture uses self-attention to process input sequences in parallel - Encoder blocks process input sequences bidirectionally (BERT-style) - Decoder blocks generate output sequences autoregressively (GPT-style) - Encoder-decoder models handle sequence-to-sequence tasks (T5-style) - Layer normalization and residual connections stabilize training - Feed-forward networks (FFN) follow attention layers in each block

📖 Attention Is All You Need - Original Paper - Foundational transformer paper 📖 NVIDIA NeMo Framework - NVIDIA's framework for building LLMs 📖 Hugging Face Transformers - Open-source transformer library

1.2 Attention Mechanisms¶

Self-Attention: - Query (Q), Key (K), Value (V) matrices derived from input embeddings - Attention score = softmax(QK^T / sqrt(d_k)) * V - Multi-head attention runs multiple attention computations in parallel - Each head can learn different relationship patterns - Typical head counts: 12 (base models), 32-96 (large models)

Attention Variants: - Causal/masked attention - prevents attending to future tokens (decoder) - Cross-attention - attends between encoder and decoder representations - Grouped-query attention (GQA) - shares KV heads for efficiency - Multi-query attention (MQA) - single KV head shared across all query heads - Flash Attention - memory-efficient attention computation

📖 NVIDIA Megatron-LM - Large-scale model training 📖 Flash Attention Paper - IO-aware exact attention algorithm

1.3 Tokenization¶

Methods: - Byte-Pair Encoding (BPE) - iteratively merges frequent character pairs; used by GPT models - WordPiece - similar to BPE but uses likelihood-based merging; used by BERT - SentencePiece - language-agnostic tokenizer; used by LLaMA, T5 - Unigram - probabilistic tokenizer that removes tokens from vocabulary

Important Concepts: - Vocabulary size affects model capacity and efficiency (typically 32K-128K tokens) - Subword tokenization handles out-of-vocabulary words - Special tokens: [CLS], [SEP], [PAD], [MASK], BOS, EOS - Tokenization impacts context window utilization - Different languages have different token efficiencies

📖 Hugging Face Tokenizers - Fast tokenizer library 📖 SentencePiece - Unsupervised text tokenizer

1.4 Pre-training and Scaling¶

Pre-training Objectives: - Causal Language Modeling (CLM) - predict next token (GPT-style) - Masked Language Modeling (MLM) - predict masked tokens (BERT-style) - Span Corruption - predict masked spans (T5-style)

Scaling Laws: - Model performance scales predictably with compute, data, and parameters - Chinchilla scaling - optimal ratio of data tokens to model parameters - Emergent abilities appear at certain scale thresholds - Larger models are more sample-efficient but require more compute

📖 NVIDIA Nemotron Models - NVIDIA's model family 📖 Scaling Laws Paper - Neural language model scaling laws

Domain 2: Prompt Engineering (20%)¶

This domain covers techniques for designing effective prompts and controlling LLM behavior.

2.1 Prompting Techniques¶

Zero-Shot Prompting: - Providing only the task instruction without examples - Works well for tasks the model was trained on - Example: "Summarize the following text in 3 sentences:"

Few-Shot Prompting: - Providing examples of input-output pairs before the task - Improves performance on novel or complex tasks - Typically 2-5 examples are sufficient - Example order and quality matter significantly

Chain-of-Thought (CoT) Prompting: - Encouraging step-by-step reasoning in the response - "Let's think step by step" - simple CoT trigger - Particularly effective for math, logic, and multi-step reasoning - Self-consistency - sampling multiple CoT paths and taking majority vote

📖 NVIDIA AI Playground - Interactive model testing 📖 Prompt Engineering Guide - NVIDIA prompt engineering blog

2.2 Advanced Prompting Strategies¶

System Prompts: - Set model behavior, personality, and constraints - Define output format and response style - Establish safety boundaries and topic restrictions

Output Control: - Temperature (0.0-1.0+): controls randomness of outputs - Low (0.0-0.3): deterministic, factual tasks - Medium (0.4-0.7): balanced creativity - High (0.8-1.0+): creative, diverse outputs - Top-k: limits token selection to k most likely tokens - Top-p (nucleus sampling): limits tokens to cumulative probability p - Max tokens: controls response length - Stop sequences: define termination points

Structured Output: - JSON formatting instructions - XML/HTML structured responses - Markdown table generation - Function calling and tool use patterns

📖 NVIDIA NIM API Reference - Model API parameters 📖 NVIDIA Prompt Engineering Tutorial - LLMOps and prompt management

2.3 Prompt Security¶

Prompt Injection: - Adversarial inputs that override system instructions - Direct injection - user input contains new instructions - Indirect injection - malicious content in retrieved documents

Mitigation Strategies: - Input validation and sanitization - Clear separation between system and user content - Output filtering and validation - NeMo Guardrails for programmable safety

📖 NVIDIA NeMo Guardrails - Safety toolkit for LLM applications

Domain 3: RAG and Vector Databases (20%)¶

This domain covers retrieval-augmented generation architecture and vector database fundamentals.

3.1 RAG Architecture¶

Components: 1. Document Ingestion - loading and processing source documents 2. Chunking - splitting documents into manageable pieces 3. Embedding - converting text chunks to vector representations 4. Indexing - storing embeddings in a vector database 5. Retrieval - finding relevant chunks for a given query 6. Generation - using retrieved context to generate responses

Why RAG: - Grounds LLM responses in specific, up-to-date information - Reduces hallucination by providing factual context - Avoids expensive model retraining for new knowledge - Enables domain-specific answers without fine-tuning - Provides source attribution for generated content

📖 NVIDIA NeMo Retriever - NVIDIA RAG components 📖 NVIDIA RAG Pipeline Tutorial - Building RAG with NVIDIA tools 📖 NVIDIA AI Blueprints - Reference architectures for RAG

3.2 Embedding Models¶

Key Concepts: - Embedding models convert text to dense vector representations - Dimensionality typically ranges from 384 to 4096 dimensions - Similarity measured via cosine similarity, dot product, or Euclidean distance - Bi-encoder models encode query and document independently - Cross-encoder models score query-document pairs jointly (re-ranking)

Popular Embedding Models: - NVIDIA NV-Embed - NVIDIA's embedding models - E5 / BGE - open-source embedding models - OpenAI text-embedding - commercial embedding API - Sentence-BERT - foundational sentence embedding approach

📖 NVIDIA Embedding Models - NVIDIA embedding model catalog 📖 MTEB Benchmark - Embedding model evaluation

3.3 Vector Databases¶

FAISS (Facebook AI Similarity Search): - Open-source library for efficient similarity search - Index types: Flat (exact), IVF (inverted file), HNSW (graph-based) - GPU-accelerated search for large-scale deployments - Best for: batch processing, research, GPU-accelerated search

Milvus: - Purpose-built open-source vector database - Supports multiple index types and distance metrics - Horizontal scaling and high availability - Best for: production deployments, large-scale applications

Other Vector Databases: - Pinecone - fully managed vector database service - Weaviate - open-source with hybrid search - Chroma - lightweight, developer-friendly - Qdrant - high-performance with filtering

📖 Milvus Documentation - Milvus vector database docs 📖 FAISS Documentation - FAISS library documentation

3.4 Chunking and Indexing Strategies¶

Chunking Methods: - Fixed-size - split by character/token count with overlap - Recursive - split by multiple separators hierarchically - Semantic - split based on topic/meaning boundaries - Document-aware - split respecting document structure (headers, paragraphs)

Chunk Size Considerations: - Smaller chunks (128-256 tokens) - more precise retrieval, less context - Larger chunks (512-1024 tokens) - more context, potentially less precise - Overlap (10-20%) prevents information loss at boundaries - Optimal size depends on document type and query patterns

Indexing Methods: - Flat - exact search, highest accuracy, O(n) search time - IVF (Inverted File) - partition-based approximate search - HNSW (Hierarchical Navigable Small World) - graph-based ANN search - PQ (Product Quantization) - compressed vectors for memory efficiency

📖 LangChain Text Splitters - Chunking implementations 📖 LlamaIndex Documentation - RAG framework documentation

Domain 4: Fine-Tuning and Customization (15%)¶

This domain covers methods for adapting pre-trained models to specific tasks and domains.

4.1 Full Fine-Tuning¶

Process: - Update all model parameters on task-specific data - Requires significant GPU memory (full model + optimizer states + gradients) - Risk of catastrophic forgetting of pre-trained knowledge - Best for: large datasets, significant domain shift, when resources allow

Requirements: - GPU memory roughly 4-6x model parameter size (FP16 + optimizer states) - High-quality labeled training data - Careful hyperparameter selection (learning rate, epochs, batch size) - Validation set for monitoring overfitting

📖 NVIDIA NeMo Framework Training - LLM training with NeMo 📖 NVIDIA NeMo Customization Guide - Model customization workflows

4.2 Parameter-Efficient Fine-Tuning (PEFT)¶

LoRA (Low-Rank Adaptation): - Freezes original model weights - Adds small trainable low-rank matrices (A and B) to attention layers - Typical rank: 8-64 (higher rank = more capacity but more parameters) - Reduces trainable parameters by 90-99% - Can be merged into base model for zero-overhead inference - Multiple LoRA adapters can be swapped for different tasks

QLoRA: - Combines LoRA with 4-bit model quantization - Enables fine-tuning 65B+ models on single consumer GPUs - Uses NF4 (Normal Float 4-bit) quantization - Double quantization for additional memory savings - Paged optimizers to handle memory spikes

Other PEFT Methods: - Prefix Tuning - prepends trainable vectors to each layer - P-Tuning v2 - learnable continuous prompts at every layer - Adapter Layers - inserts small networks between transformer layers - IA3 - scales activations with learned vectors (very few parameters)

📖 NVIDIA NeMo PEFT - PEFT with NeMo Framework 📖 LoRA Paper - Original LoRA research 📖 QLoRA Paper - Quantized fine-tuning research 📖 Hugging Face PEFT Library - PEFT implementation library

4.3 Alignment Methods¶

Supervised Fine-Tuning (SFT): - Train on high-quality instruction-response pairs - First step in most alignment pipelines - Data quality is more important than quantity

RLHF (Reinforcement Learning from Human Feedback): 1. Train a reward model from human preference data 2. Use PPO to optimize the LLM against the reward model 3. KL divergence penalty prevents deviation from base model 4. Expensive and complex to implement

DPO (Direct Preference Optimization): - Simplifies RLHF by eliminating the reward model - Directly optimizes on preference pairs (chosen vs rejected) - More stable training than RLHF - Growing in popularity due to simplicity

📖 NVIDIA NeMo Alignment - RLHF/DPO with NeMo 📖 InstructGPT Paper - RLHF alignment methodology

4.4 Training Data Best Practices¶

Data Quality: - Clean, diverse, and representative data - Balanced distribution across task types - Consistent formatting and annotation standards - Minimum 1,000-10,000 examples for meaningful fine-tuning

Data Formats: - Instruction format: system prompt + user instruction + assistant response - Chat format: multi-turn conversation with roles - Preference format: chosen and rejected response pairs (for DPO)

📖 NVIDIA NeMo Data Curator - Data preparation tools

Domain 5: Deployment and Inference (10%)¶

This domain covers deploying LLMs to production with optimized performance.

5.1 NVIDIA NIM¶

Overview: - Pre-optimized inference containers for NVIDIA GPUs - OpenAI-compatible API endpoints - Supports popular model architectures - Automatic hardware optimization - Simple deployment with Docker containers

Key Features: - Built on TensorRT-LLM for maximum performance - Includes optimized tokenization and pre/post-processing - Health checks and monitoring endpoints - Horizontal scaling support - Model caching and warm-up

📖 NVIDIA NIM Documentation - Complete NIM guides 📖 NVIDIA NIM for LLMs - LLM-specific NIM documentation 📖 NVIDIA AI Enterprise NIM - Enterprise NIM deployment

5.2 TensorRT-LLM¶

Optimization Techniques: - Graph optimization and kernel fusion - Mixed-precision execution (FP16, BF16, FP8, INT8, INT4) - Paged KV-cache for memory-efficient attention - In-flight batching (continuous batching) for throughput - Tensor parallelism across multiple GPUs - Pipeline parallelism for very large models

Quantization Methods: - FP8 - 8-bit floating point (H100+ GPUs) - INT8 - SmoothQuant for weight and activation quantization - INT4 - AWQ (Activation-aware Weight Quantization) - GPTQ - Post-training quantization with calibration data - W4A16 - 4-bit weights, 16-bit activations

📖 TensorRT-LLM Documentation - Complete TensorRT-LLM guides 📖 TensorRT-LLM GitHub - Source code and examples

5.3 Triton Inference Server¶

Features: - Multi-framework support (TensorRT, PyTorch, TensorFlow, ONNX) - Dynamic batching for throughput optimization - Model ensembles and pipelines - Metrics and monitoring (Prometheus-compatible) - Model versioning and A/B testing

📖 Triton Inference Server - Triton documentation 📖 Triton Model Analyzer - Performance profiling

5.4 Inference Optimization Concepts¶

Batching Strategies: - Static batching - fixed batch size, all requests processed together - Dynamic batching - groups requests within time window - Continuous/in-flight batching - adds new requests as slots free up - Continuous batching provides best throughput for LLMs

KV-Cache: - Stores computed key/value pairs to avoid recomputation - Grows linearly with sequence length and batch size - Paged attention (vLLM-style) manages cache like virtual memory - KV-cache size is often the memory bottleneck for long contexts

Speculative Decoding: - Uses smaller draft model to generate candidate tokens - Larger model verifies candidates in parallel - Speeds up generation without quality loss - Effective when draft model is fast and accurate

📖 vLLM Documentation - PagedAttention and efficient serving

Domain 6: Ethics and Responsible AI (10%)¶

This domain covers responsible development and deployment of generative AI systems.

6.1 Bias and Fairness¶

Sources of Bias: - Training data reflects societal biases - Underrepresentation of certain groups in data - Annotation bias from human labelers - Amplification of existing biases during training

Mitigation Approaches: - Diverse and representative training data - Bias evaluation benchmarks and metrics - Debiasing techniques during and after training - Regular auditing of model outputs

📖 NVIDIA AI Responsibility - NVIDIA's responsible AI practices 📖 NVIDIA Trustworthy AI - Blog posts on AI ethics

6.2 NeMo Guardrails¶

Capabilities: - Programmable rules for LLM input/output control - Topical rails - restrict conversation to allowed topics - Safety rails - block harmful, toxic, or inappropriate content - Fact-checking rails - verify claims against knowledge bases - Jailbreak detection - identify attempts to bypass safety

Implementation: - Colang language for defining conversational flows - Configuration-based rail definitions - Integration with existing LLM applications - Customizable for domain-specific requirements

📖 NeMo Guardrails Documentation - Complete guardrails guide 📖 NeMo Guardrails GitHub - Source code and examples

6.3 Hallucination and Safety¶

Hallucination Types: - Factual hallucination - generating false facts - Faithfulness hallucination - contradicting provided context - Fabrication - creating non-existent references or sources

Mitigation Strategies: - RAG to ground responses in verified sources - Temperature reduction for factual tasks - Source attribution and citation requirements - Human-in-the-loop verification for critical applications - Confidence scoring and abstention for uncertain outputs

6.4 Privacy and Governance¶

Data Privacy: - PII detection and redaction in inputs/outputs - Data retention and deletion policies - Consent management for training data - Compliance with GDPR, CCPA, and other regulations

AI Governance: - Model documentation and model cards - Version tracking and reproducibility - Access controls and audit logging - Incident response procedures

📖 EU AI Act Overview - European AI regulation 📖 NIST AI Risk Management Framework - US AI risk framework

Quick Reference Tables¶

Model Architecture Comparison¶

PEFT Method Comparison¶

Quantization Precision Comparison¶

NVIDIA Tool Selection Guide¶

Exam Day Tips¶

Before the Exam¶

• Review the NVIDIA tool ecosystem and when to use each component
• Ensure you understand transformer architecture at a conceptual level
• Practice distinguishing between fine-tuning methods and their trade-offs
• Review RAG pipeline components and chunking strategies
• Refresh knowledge of responsible AI practices

During the Exam¶

• Read each question carefully - look for NVIDIA-specific keywords
• Eliminate obviously wrong answers first
• When unsure, consider what NVIDIA would recommend as best practice
• Don't overthink - the associate level tests foundational knowledge
• Manage time carefully with approximately 1 minute per question

Key Differentiators to Remember¶

• NIM = deployment/serving, NeMo = training/customization
• LoRA = efficient fine-tuning, QLoRA = LoRA + quantization
• RAG = external knowledge, fine-tuning = embedded knowledge
• Temperature controls randomness, top-p controls diversity
• Encoder = understanding tasks, decoder = generation tasks