Deepseek OCR

What is DeepSeek OCR?

DeepSeek OCR is a cutting-edge optical character recognition (OCR) system designed to extract text from images, PDFs, and handwritten notes with remarkable precision. Unlike traditional OCR models that struggle with accuracy and layout understanding, DeepSeek OCR excels at reading, understanding, and converting visual text into digital text. It uses a unique approach called context optical compression, which translates text into a visual representation, significantly reducing the number of tokens needed to represent a page of text.

Benefits

DeepSeek OCR offers several key advantages:

• High Accuracy: It reads and understands visual text with extraordinary precision, making it ideal for extracting text from complex documents.
• Efficient Compression: By converting text into a visual representation, DeepSeek OCR reduces the number of tokens needed, making it more efficient than traditional OCR models.
• Versatility: It can handle a wide range of documents, including handwritten notes, PDFs, and images with messy layouts.
• Multi-Resolution Support: DeepSeek OCR supports multiple input resolutions, allowing users to choose between different levels of detail and compression.

Use Cases

DeepSeek OCR can be used in various scenarios, including:

• Document Digitization: Converting physical documents, such as receipts, invoices, and contracts, into digital text.
• Data Extraction: Extracting text from images and PDFs for further processing and analysis.
• Handwritten Text Recognition: Recognizing and converting handwritten notes into digital text.
• Medical and Legal Documents: Handling complex documents with intricate layouts and handwritten text.

How to Access DeepSeek OCR

To use DeepSeek OCR, you need to install the necessary libraries and load the model. Here are the steps:

1. Install the required libraries:

!pip install torch==2.6.0 torchvision==0.21.0 torchaudio==2.6.0 --index-url https://download.pytorch.org/whl/cu118!pip install flash-attn!pip install transformers==4.46.3!pip install accelerate==1.1.1!pip install safetensors==0.4.5!pip install addict

2. Load the model:

from transformers import AutoModel, AutoTokenizerimport torchmodel_name = "deepseek-ai/DeepSeek-OCR"tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)model = AutoModel.from_pretrained(model_name, trust_remote_code=True, use_safetensors=True)model = model.eval().cuda().to(torch.bfloat16)

3. Use the model to process documents:

prompt = "<image><|grounding|>Convert the document to markdown. "image_file = '/content/img_1.png'output_path = '/content/out_1'res = model.infer(tokenizer, prompt=prompt, image_file=image_file,output_path=output_path, base_size=1024, image_size=640,crop_mode=True, save_results=True, test_compress=True)

Performance and Benchmarks

DeepSeek OCR achieves outstanding rates of compression and OCR accuracy. It demonstrates good text retention even at increased levels of compression, achieving over 96% accuracy at 10x compression and sustaining around 85-87% accuracy at 15-20x compression. On the OmniDocBench, DeepSeek OCR surpasses leading OCR models and vision language models, achieving an Edit Distance (ED) of less than 0.25, which is almost human-level accuracy.

Conclusion

DeepSeek OCR sets a new standard for OCR technology by significantly reducing token usage while retaining most of the information. It is open-source and available for developers to experiment with. This innovative approach to text representation could have significant implications for AI memory and context management, making it a valuable tool for various applications.