Abstract
Post-training quantization has become essential for deploying Large Language Models (LLMs) on resource-constrained hardware. This project explores the use of the Posit number system as a quantization target for LLMs, investigating its potential as an alternative to conventional formats such as INT8, FP8, NVFP4. A range of open-source LLMs will be quantized and systematically compared under standardized evaluation benchmarks.
Tasks
- Survey existing quantization methods and the Posit number format; identify relevant open-source LLMs as quantization targets.
- Implement Posit-based quantization and apply it across selected LLMs.
- Evaluate and compare quantized models on standardized benchmarks against established quantization formats (INT8, FP8, NVFP4).
- Analyze results and derive insights on the viability of Posit quantization for efficient LLM inference.
Requirements
- Basic understanding of deep learning and transformer-based LLMs.
- Some experience with Python; familiarity with PyTorch is a plus.
- Curiosity about quantization concepts (PTQ, calibration, per-group scaling) — no prior expertise required.
- Interest in low-level numeric formats and hardware-aware machine learning.
- Most importantly: a genuine interest in this topic and the enthusiasm to dive into and learn these areas.
How to apply
- Please send an email to yue.wu(at)fau.de
- Include a short motivation letter, your CV, and the transcript of your current degree program.