Which is the best LLM
Which is the best LLM? Clearly a loaded topic with lots of strong opinions. The best LLM is probably one you've never heard of...yet... but brings all the pieces together. Let me explain:
Most LLM evaluations are academically interesting but of little practical application. I've seen LLM researchers get excited because can LLM solved some obscure logic problem or could parse a 1,000-page textbook and get high scores on a professional exam. I understand that these are all tests of the underlying LLMs competence, but I don't know if any of that prowess translates into practical consumer-centric utility. To me, the bet LLM is not one that scores x% on [insert professional exam here] but is one that has the following attributes:
1. Is always with you: no brainer it has to be on your phone - the device you are most likely to have on you all day.
2. Understands your context: knows your - time zone, location, contacts, preferences, calendar, call log, browsing history, messaging,
3. Is private (i.e. runs on device): most of us shudder at the thought of sharing all of the above context online - not just because it could be controlled by one company but the inherent security risks of sharing all of this highly personal information.
So, what would the technical requirements be to implement the above? You would need a:
1. Powerful phone with processing power to spare.
2. A single service that brings together your location, contacts, preferences, calendar, call log, browsing history, messaging.
3. A company that is not incentivized to monetize your personal information i.e. does not earn its money from ads.
There is only one company that meets all of these requirements today - Apple. I had written an earlier post about how Apple could leverage their silicon chips for on device LLMs https://lnkd.in/gfrcRTH5
Apple acquired over 30 AI startups in 2023 - many focused on smaller and nimbler implementations of AI on low powered devices. This indicates that its not LLM processing power that Apple is chasing but the ability to leverage their rich device ecosystem powered by Apple Silicon. the best LLM may just be a version of Siri but with an LLM running on device on the backend. As a customer here are things I want to be able to do and would be useful:
1. When did I last eat at this restaurant and who was I with? What did I order? (Services sed: Location, calendar, contacts, Wallet)
2. I want to talk with Person X sometime next week (Calendar - figure out time zones, contacts, meeting preference times)
3. After which exercise did I feel the most energetic and focused? (Apple watch, fitness, + heart rate variability insights from a company like Welltory)
