Hugging Face Opens Up Carbon-X, a 14.8B Language Model That Reads and Writes DNA
The HuggingFaceBio team has released Carbon-X, an open 14.8B-parameter decoder-only language model trained jointly on DNA sequences and natural-language text. Users can drop raw DNA into an ordinary chat conversation and the model answers in English: it predicts whether mutations are benign or pathogenic, identifies regions that control gene activity, classifies sequences by organism group, predicts the next bases of a sequence, and explains its own reasoning.
- Open 14.8B-parameter model trained on DNA and natural language, fine-tuned from Qwen3-14B
- Hybrid tokenizer: the text BPE vocabulary plus 5,515 DNA tokens, one token covering 6 base pairs
- Averages 66.7% across five DNA benchmarks, 27 points above its base model, ahead of every open model evaluated
- Close to GPT-6.1 Sol (67.5%) at about 1/1,800 of the inference cost per problem
- Analyzing all 2.4 million ClinVar variants of uncertain significance estimated at about $300
- Weights fit on a single 80GB GPU, keeping sensitive sequences on researchers’ own infrastructure
One decoder for DNA and text
Architecturally, Carbon-X shares a single token stream and a single decoder across DNA and natural language, with no separate DNA encoder. It uses a hybrid tokenizer: text keeps the Qwen3 BPE vocabulary while DNA is encoded as non-overlapping 6-mers, meaning one DNA token covers six base pairs. The model has 40 layers, a hidden size of 5120, GQA attention (40 query heads, 8 KV heads), and a 40,960-token context window; it was post-trained and evaluated with thinking disabled, answering directly rather than through long reasoning chains.
Performance and cost
Across five DNA benchmarks covering variant-effect prediction, sequence understanding and next-base prediction, Carbon-X averages 66.7%, 27 points above its Qwen3-14B base and ahead of every open model the team evaluated. It comes within a point of GPT-6.1 Sol at 67.5%, at roughly 1/1,800 of the inference cost per problem. The team estimates that analyzing the 2.4 million variants of uncertain significance in ClinVar would cost about $300 with Carbon-X, compared with $32,000 to $4 million for the frontier models evaluated.
Deployment flexibility is the other selling point: the weights fit on a single 80GB GPU in bfloat16, which means research involving sensitive or unpublished sequences can run on researchers’ own infrastructure without sending data to the cloud. The model card also includes a limitations section, and research use should be assessed against the official documentation.
Source: HuggingFaceBio model card
(Cover image is AI-generated, for illustrative purposes only)
