About Us
What is AI Model Review?
AI Model Review is dedicated to providing transparent reviews and information about generative AI models.
We are currently in the midst of an AI explosion with new generative AI models being released frequently, and by a wide range of organizations, from government institutions, large dominant tech companies to individual creators.
For consumers like us, it can be daunting to understand what the different models have to offer.
While there are information sources about model performance and model rankings , we aim to provide a more qualitative perspective, by asking the models to respond to a standardized set of questions. This way we can directly compare how the models respond and get an intuition about the strengths and weaknesses of different models.
What AI Models Are Available to Compare?
There are currently over 75 models available to compare, and we will continue to add new models as they become available. With the goal of including as many models as possible. Our comparisons include both: models that can only be accessed via a service such as ChatGPT and models that can be downloaded to run locally like Llama. In the model comparison cards we provide links that will take you to the model page where you can download the weights for the locally available models.
How is Data for the Model Comparisons Generated?
We take our standard set of questions and prompt the models 4 times for each question. Each question is prompted with a different temperature setting. We want to see how the temperature setting affects the model’s response. You can change the temperature by selecting from the dropdown at the bottom of the model comparison card to see the different responses.
What Does the Newsletter Provide?
The AI Model Review Newsletter is the best way to stay informed and get updates on:
- New models added to our comparison page
- New model reviews and articles
- New categories and questions added to our test set
- The latest news from the world of Generative AI
- AI tools and techniques to get the most from these technologies
Why do I get different responses when I try these prompts?
You may get slightly different results when you try these prompts yourself, this may be because:
- Generative AI is not completely deterministic, submitting the same prompt repeatedly can give slightly different results each time.
- When you access the models through a chat interface on the model provider’s websites, there are instructions that the provider injects into the prompt for various reasons. These instructions change the behavior of the model to be more aligned with the provider's wishes. All of the testing done at AIModelReview uses the API for the model rather than the chat interface, so those extra instructions are not included.
- By using prompt engineering techniques you can get the models to answer some of the tricky prompts correctly, where they might not have answered correctly with our basic prompts. Here at AI Model Review our goal is to see how models respond out of the box. We want to see how the models compare on basic tasks without the users having to put a lot of effort into getting the model to respond correctly. We believe that a model that performs strongly with basic prompts will continue to be stronger with prompt engineering methods applied verses a model which is weaker with the basic prompts.
Why should I trust AI Model Review?
We are not AI researchers, AI developers or affiliated with any organizations which develop AI technologies. We are avid enthusiasts and consumers of AI technology and believe these tools will empower all of us to reach our full potential in productivity and creativity.
We provide model responses from all of our tests openly on our model comparison page so you do not need to rely on trust alone to get a good understanding of the capabilities of AI models. While our reviews are written from our own perspective, our perspective is that of the average consumer. Those who share in this perspective may find value in our reviews.
How can I get in touch?
We welcome comments, suggestions and feedback. Please feel free to contact us via email at AIMR@AIModelReview.com