AI: LLaMA Going APE
A new hot thing: What happens when you cross a llama with an ape? When the llama is Meta’s LLaMA 2 LLM and the ape is Justine Tunney’s ‘Actually Portable Executable’ (Cosmopolitan libc) format, what results is ‘llamafile’, a single-file executable that will run on pretty much any personal computer system, delivering a web server with an LLM web application already serving. The LLM will default to CPU inference, but there look to be reasonably uncomplicated instructions on how to get it to use a GPU in your system. And it looks like the various licenses do not actually exclude using it for commercial purposes (though Meta does have a monthly users threshold of 700M where one has to then apply for a specific license for LLaMA 2).
A write-up on installation and use:
https://simonwillison.net/2023/Nov/29/llamafile/
As someone noted, this means that you can fit an AI large language model (LLM) onto a flash drive, and reasonably expect it will just work on whatever system you have at hand, with no requirement for a network connection and no real distinction on whether you are running a particular operating system.
If the AI cat was not out of the bag before, this will certainly do it.
