AI News #16

Here’s what caught our eye this past week.

FeatUp

  • Use a multiview reconstruction loss to learn to upscale image features, for greater accuracy in downstream tasks like segmentation and depth estimation.
  • Paper


EnvGen

  • Use an LLM to generate environments that help RL models learn better.
  • Paper
  • Tweet


RAFT

  • Finetune the LLM to ignore irrelevant information retrieved during RAG.
  • Paper


When Do We Not Need Larger Vision Models?

  • Use multi-resolution image features for improved vision and multimodal performance.
  • Paper
  • Code
  • Tweet


Logits of API-Protected LLMs Leak Proprietary Information

  • Extract information about an LLM’s architecture using just the output vector (logits).
  • Paper


Evolutionary Optimization of Model Merging Recipes

  • Uses evolutionary algorithms to merge the weights and architectures of foundation models.
  • Paper


SceneScript

  • Given a video of an environment, SceneScript outputs the commands necessary to make a 3D representation of the environment.
  • Paper
  • Announcement


LLM2LLM

  • Use a large LLM to generate synthetic data tailored to help a smaller LLM address its weak spots during training.
  • Paper
  • Code


RewardBench

  • A new benchmark for evaluating reward models, in the context of Reinforcement Learning from Human Feedback (RLHF).
  • Code
  • Paper


Reverse Training to Nurse the Reversal Curse


Vid2Robot

  • A new method for generating robotic actions directly from videos of tasks.
  • Project page
  • Paper


DROID

  • A new robot manipulation dataset with 350 hours of interaction data.
  • Project page


MindEye2


PERL

  • Use LoRA to train the reward model and policy model in RLHF.
  • Paper


Grok-1

Inflection AI CEO leaving for Microsoft

  • A co-founder of Inflection AI is leaving for Microsoft.
  • Inflection AI raised $1.3 billion last June.
  • Announcement

Stable Video 3D:

  • Stability AI releases Stable Video 3D, a new image-to-3D model.
  • Announcement

Runway + Musixmatch

  • Partnership to allow artists to make lyric music videos using Runway.
  • Announcement

Suno v3

  • Startup releases a new version of their AI music generator. It’s safe to say musicians are here to stay, for now.
  • Announcement + sample

Stay up to date

Interested in future weekly updates? Stay up to date by joining our Slack Community!

Recent Posts