How to Summarize PDFs with AI (March 2024)

In this article, I will test and compare a few specific methods for summarising PDFs with AI, using some of the most popular AI chatbots out there. I work as a writer and editor and often find it useful to summarise articles quickly for research purposes.

‍

These methods could be helpful if you're a researcher, student, journalist, or any type of content creator looking to make your handling of PDFs more efficiently.

‍

Gone are the days when we had to manually go through piles after piles of documents just to extract information. AI can now help us streamline this task and focus more on the more important parts of our day.

‍

So how do these chatbots manage to do this? Basically, they use large language models (LLMs), AI models that can process and respond to human language or text. An important characteristic of an LLM for summarising loads of information like this is its context window; it controls how much text the AI can understand at any one time. The higher the context window, the more information an LLM can understand. For example, the chatbots in this test have the following context windows: ChatGPT Plus (with GPT-4 Turbo) has a 128k tokens context window, the same goes for Gemini Pro 1.5, and Claude 2.1 has 200k. This isn't the only important component that makes an LLM useful for summarisation, but it might be the one with the most influence on the outcome (we'll test this in a minute!).

‍

It's also good to be aware that there are some common challenges users may face when using these AI tools for summarization. For one, LLMs are often more effective at recalling the beginning and last parts of the document, while the stuff in the middle often gets forgotten. Some have also experienced biased summaries or hallucinations, especially for long documents.

‍

Next, I'll do an informal test of different methods and tools to see which AI model will provide the best and most comprehensive summary for a 33-page sample document for Disney's Q1 FY24 Earnings Conference Call. I will be looking at each AI model’s efficiency, accuracy, and range when it comes to summarizing PDFs using three questions that cover the beginning, middle and last parts of the document:

‍

• How did Disney's entertainment direct‐to‐consumer operating income perform this quarter of 2024 compared to 2023?
• What is Disney's target annualized cost target for this year?
• What was the most successful video game title from Disney in 2023?

‍

I'll include three of the most popular AI chatbots for testing: ChatGPT, Claude and Gemini. For ChatGPT, since I already have a Plus subscription (costs $20/month) and have access to custom GPTs, I’ll also test summarising the PDFs using two GPTs dedicated to this task. For Claude and Gemini, I'll use the free versions in my test.

‍

So, whether you’re a student looking for better (and free) ways to study your books or a professional who needs summaries of their company’s reports, this guide will give you a clear understanding of which AI will best suit your summarization needs.

‍

‍

Our top choice

Out of all the AI tools I’ve tested to summarize PDF documents, Claude AI topped my list for its exceptional ability to provide detailed and comprehensive results.

‍

While it’s true that the other chatbots and GPTs I tested generated useful summaries, often with better readability, Claude AI exceeded them by going beyond the surface and including in-depth information from the PDF. What’s surprising is that this performance is a result of their free model; imagine what more it could do with its premium model.

‍

Method 1: Summarise PDF files with ChatGPT

‍

‍

First up in my test is the most popular chatbot of them all, ChatGPT. Developed by OpenAI, this chatbot excels at many tasks, and its updated model (GPT-4) is widely considered the best option for most tasks that require advanced logic and reasoning. From answering questions and writing an essay to teaching you how to code or finish your assignments, anything you need, this powerful AI can do for you. While the free version (GPT-3.5) offers a good glimpse into what ChatGPT is really capable of, the real horsepower lies in the paid version and the GPT-4 model it gives access to.

‍

To see this for myself, I used the premium version of ChatGPT, called ChatGPT Plus, for this test. As expected, it managed to provide a comprehensive and accurate summary of the sample PDF file. The event title, date and name of the participants were mentioned correctly. The summary also captured a lot of important information from the event. However, this information can be easily located in the introduction part of the document. 

‍

Additionally, the summary was missing a lot of important numerical data presented from the earnings call. To extract this, you will need to ask further questions, just like I did, and fortunately, it was reliable enough to provide the data I needed. Although, you might need to rephrase and reword your questions a couple of times before it can understand what specific data you are asking for.

‍

Method 2: Using GPTs inside ChatGPT

With the recent rollout of custom GPTs, you can now use tailored chatbots created by other users for different tasks.

‍

GPT 1. AI PDF

‍

‍

‍

As mentioned, ChatGPT has been a household name when it comes to artificial intelligence. Early this year, they fully rolled out custom GPTs, a feature under ChatGPT that allows users to create or modify chatbots to perform specific tasks. The first GPT I will test is AI PDF, which not only has the ability to summarize PDFs but is a more general tool for storing, searching and interacting with your PDFs.

‍

This GPT successfully generated a clear and quick summary of the PDF file provided. Most of the information in the generated summary was accurate. However, it only repeated the key highlights mentioned by one of the speakers, which can be immediately seen on the second and third pages of the file.

‍

Further numerical data was also missing, and while it mentioned the stock buyback, it was clear that this information could again be easily found on the third page of the document. Just like when using default ChatGPT without a GPT, I had to ask this AI PDF questions for the data I needed and it did so perfectly. Based on the test I did, it delivered a shallow summary compared to some of the other methods in this test, so users requiring thorough summarization might be better off using another method.

‍

GPT 2. AskYourPDF Research Assistant

‍

‍

‍

AskYourPDF Research Assistant is another custom GPT that is capable of helping users look for information by manipulating several documents and files.

‍

Unlike the first two tests I did, this GPT provided a summary of the sample PDF document in a bulleted style format, offering better readability for users, which I personally liked as opposed to several blocks of paragraphs. However, this GPT only summarized the overview of the announcements reported by one of the speakers from the first few pages of the document (which was, on its own, already a summary of the event).

‍

Clearly, it was not as extensive as I would have liked, as it did not seem to have dived deeper into the sample file for the summary.

‍

Method 3: Summarise PDF files with Claude AI (Anthropic)

‍

‍

‍

Claude AI is another top contender in the world of AI chatbots. This AI model was developed by Anthropic, an organization that focuses on AI safety and research. They are currently offering three models, and for my test, I will be using its free model called Claude Sonnet. If you want to get access to their most intelligent model called Claude Opus, you may purchase it for $20 per month. Even though Claude Sonnet is free to use, you will be required to register on their website using your email and phone number. This might be a problem for users who do not wish to share their personal contact information. 

‍

This AI chatbot surprised us with a much more thorough summary of the PDF file compared to ChatGPT Plus. Even though numerical information and data were also missing, this AI was listing down a lot of information that can only be found once you read the entire article yourself. It was even mentioning the Q&A part of the event which was not covered by any of the other methods above.

‍

When it came to asking for specific information and data, it directly referenced what the speakers said (in quotes), so you know that it was giving factual and reliable data. I tested whether it could answer my questions, and it provided precise data and summaries for the three questions. And did I mention that they’re also dyslexia-friendly? If you are struggling with this learning disability, you may edit Claude AI’s fonts to be “dyslexic friendly” for a more accessible experience, at no cost.

‍

Method 4: Summarise PDF files with Gemini (Google)

‍

‍

‍

Last but not least, I’ve also included Google’s AI chatbot for the test. Gemini, formerly known as Google Bard, is a powerful AI model upgraded in terms of performance and is also multimodal. For this experiment, I will utilize the free version of Gemini to summarize PDFs, but if you want to experience how powerful their top-tier model is, you may subscribe to Gemini Advance for $19.99 per month.

‍

Just like Claude AI, Gemini will require you to register using your email to access their chatbot. Unfortunately, the free version of Gemini does not allow users to upload documents (such as PDF files). Instead, I inserted the URL for the PDF and asked the chatbot to provide a summary. Out of all the tested methods and models above, this AI gave the most brief and least informative summary of the document. The summary was a short paragraph that only mentioned what the event was about and a few of the announcements from the first few pages.

‍

Even after tweaking the prompt several times, it was still generating a very short and superficial summary. But on the bright side, it was able to provide specific data and summary for all my questions. Gemini may not be the best at summarizing but it's great at fishing out straightforward information with the right prompts.

‍

Conclusion

In this article, I tested different methods for summarising PDFs using AI. I prepared 3 specific questions and analysed the output from ChatGPT (both with and without using GPTs), Claude and Gemini.

‍

While far from a rigorous study, the aim of my little experiment was simply to test which of these options gave the best results for summarising and answering specific questions from the PDF I gave them.

‍

I’ve chosen Claude AI as the best PDF summarizing tool because it turns a lengthy and complicated document into an easy-to-understand summary. And it did so using the free version, which was wonderful.

‍

On top of that, this article has also highlighted the importance of doing your own research before committing to an AI tool for your everyday use. While there are many options available, each has its own set of pros and cons that you will need to investigate first to know which one can make your life easier when it comes to information and content.

‍