Hugging Face's logo

Rapidata

Enterprise
company
https://rapidata.ai
RapidataAI
RapidataAI
Activity Feed

AI & ML interests

RLHF, Model Evaluation, Benchmarks, Data Labeling, Human Feedback, Computer Vision, Image Generation, Video Generation, LLMs, Translations

Recent Activity

jasoncorkill  updated a dataset 4 days ago
Rapidata/2k-ranked-images-open-image-preferences-v1
maalber  updated a dataset 5 days ago
Rapidata/2k-ranked-images-open-image-preferences-v1
maalber  published a dataset 5 days ago
Rapidata/2k-ranked-images-open-image-preferences-v1
View all activity

Rapidata's activity

jasoncorkill 
posted an update 4 days ago
view post
Post
3144
🚀 We tried something new!

We just published a dataset using a new (for us) preference modality: direct ranking based on aesthetic preference. We ranked a couple of thousand images from most to least preferred, all sampled from the Open Image Preferences v1 dataset by the amazing @data-is-better-together team.

📊 Check it out here:
Rapidata/2k-ranked-images-open-image-preferences-v1

We're really curious to hear your thoughts!
Is this kind of ranking interesting or useful to you? Let us know! 💬

If it is, please consider leaving a ❤️ and if we hit 30 ❤️s, we’ll go ahead and rank the full 17k image dataset!
·
jasoncorkill 
updated a dataset 4 days ago
maalber 
updated a dataset 5 days ago
maalber 
published a dataset 5 days ago
jasoncorkill 
posted an update 5 days ago
view post
Post
2987
🔥 Yesterday was a fire day!
We dropped two brand-new datasets capturing Human Preferences for text-to-video and text-to-image generations powered by our own crowdsourcing tool!

Whether you're working on model evaluation, alignment, or fine-tuning, this is for you.

1. Text-to-Video Dataset (Pika 2.2 model):
Rapidata/text-2-video-human-preferences-pika2.2

2. Text-to-Image Dataset (Reve-AI Halfmoon):
Rapidata/Reve-AI-Halfmoon_t2i_human_preference

Let’s train AI on AI-generated content with humans in the loop.
Let’s make generative models that actually get us.
Kchanger 
published 2 datasets 7 days ago
Kchanger 
updated a dataset 7 days ago
LinoGiger 
updated a dataset 7 days ago
Kchanger 
updated a dataset 7 days ago
jasoncorkill 
posted an update 11 days ago
view post
Post
2365
🚀 Rapidata: Setting the Standard for Model Evaluation

Rapidata is proud to announce our first independent appearance in academic research, featured in the Lumina-Image 2.0 paper. This marks the beginning of our journey to become the standard for testing text-to-image and generative models. Our expertise in large-scale human annotations allows researchers to refine their models with accurate, real-world feedback.

As we continue to establish ourselves as a key player in model evaluation, we’re here to support researchers with high-quality annotations at scale. Reach out to [email protected] to see how we can help.

Lumina-Image 2.0: A Unified and Efficient Image Generative Framework (2503.21758)
LinoGiger 
updated a dataset 17 days ago
Kchanger 
updated a dataset 17 days ago
jasoncorkill 
posted an update 17 days ago
view post
Post
2249
🔥 It's out! We published the dataset for our evaluation of @OpenAI 's new 4o image generation model.

Rapidata/OpenAI-4o_t2i_human_preference

Yesterday we published the first large evaluation of the new model, showing that it absolutely leaves the competition in the dust. We have now made the results and data available here! Please check it out and ❤️ !
maalber 
updated a dataset 18 days ago
maalber 
in Rapidata/OpenAI-4o_t2i_human_preference 18 days ago

Update README.md
#1 opened 18 days ago by
maalber
jasoncorkill 
posted an update 19 days ago
view post
Post
2034
🚀 First Benchmark of @OpenAI 's 4o Image Generation Model!

We've just completed the first-ever (to our knowledge) benchmarking of the new OpenAI 4o image generation model, and the results are impressive!

In our tests, OpenAI 4o image generation absolutely crushed leading competitors, including @black-forest-labs , @google , @xai-org , Ideogram, Recraft, and @deepseek-ai , in prompt alignment and coherence! They hold a gap of more than 20% to the nearest competitor in terms of Bradley-Terry score, the biggest we have seen since the beginning of the benchmark!

The benchmarks are based on 200k human responses collected through our API. However, the most challenging part wasn't the benchmarking itself, but generating and downloading the images:

- 5 hours to generate 1000 images (no API available yet)
- Just 10 minutes to set up and launch the benchmark
- Over 200,000 responses rapidly collected

While generating the images, we faced some hurdles that meant that we had to leave out certain parts of our prompt set. Particularly we observed that the OpenAI 4o model proactively refused to generate certain images:

🚫 Styles of living artists: completely blocked
🚫 Copyrighted characters (e.g., Darth Vader, Pokémon): initially generated but subsequently blocked

Overall, OpenAI 4o stands out significantly in alignment and coherence, especially excelling in certain unusual prompts that have historically caused issues such as: 'A chair on a cat.' See the images for more examples!
  • 1 reply
·
jasoncorkill 
posted an update 28 days ago
view post
Post
3804
At Rapidata, we compared DeepL with LLMs like DeepSeek-R1, Llama, and Mixtral for translation quality using feedback from over 51,000 native speakers. Despite the costs, the performance makes it a valuable investment, especially in critical applications where translation quality is paramount. Now we can say that Europe is more than imposing regulations.

Our dataset, based on these comparisons, is now available on Hugging Face. This might be useful for anyone working on AI translation or language model evaluation.

Rapidata/Translation-deepseek-llama-mixtral-v-deepl
  • 1 reply
·
jasoncorkill 
posted an update about 1 month ago
view post
Post
2218
Benchmarking Google's Veo2: How Does It Compare?

The results did not meet expectations. Veo2 struggled with style consistency and temporal coherence, falling behind competitors like Runway, Pika, Tencent, and even Alibaba. While the model shows promise, its alignment and quality are not yet there.

Google recently launched Veo2, its latest text-to-video model, through select partners like fal.ai. As part of our ongoing evaluation of state-of-the-art generative video models, we rigorously benchmarked Veo2 against industry leaders.

We generated a large set of Veo2 videos spending hundreds of dollars in the process and systematically evaluated them using our Python-based API for human and automated labeling.

Check out the ranking here: https://www.rapidata.ai/leaderboard/video-models

Rapidata/text-2-video-human-preferences-veo2
jasoncorkill 
posted an update about 2 months ago
view post
Post
3854
Has OpenGVLab Lumina Outperformed OpenAI’s Model?

We’ve just released the results from a large-scale human evaluation (400k annotations) of OpenGVLab’s newest text-to-image model, Lumina. Surprisingly, Lumina outperforms OpenAI’s DALL-E 3 in terms of alignment, although it ranks #6 in our overall human preference benchmark.

To support further development in text-to-image models, we’re making our entire human-annotated dataset publicly available. If you’re working on model improvements and need high-quality data, feel free to explore.

We welcome your feedback and look forward to any insights you might share!

Rapidata/OpenGVLab_Lumina_t2i_human_preference