As you may know, ChatGPT collects a lot of data on the users for the improvement of their AI, but this poses risks in its own way. I was wondering whether there are privacy alternatives to ChatGPT. Perhaps on F-Droid or Aurora/PlayStore, or for Linux.
Are there any alternatives you know of? Or are there other ways to interact with ChatGPT without giving personal information, such as a privacy focussed front-end?
Ddg chat
The best option is to run them models locally. You’ll need a good enough GPU - I have an RTX 3060 with 12 GB of VRAM, which is enough to do a lot of local AI work.
I use Ollama, and my favourite model to use with it is Mistral-7b-Instruct. It’s a 7 billion parameter model optimised for instruction following, but usable with 4 bit quantisation, so the model takes about 4 GB of storage.
You can run it from the command line rather than a web interface - run the container for the server, and then something like
docker exec -it ollama ollama run mistral
, giving a command line interface. The model performs pretty well; not quite as well on some tasks as GPT-4, but also not brain-damaged from attempts to censor it.By default it keeps a local history, but you can turn that off.
Is that 12GB RAM just for the model or total?
I think the GPU has 12GB of physical ram.
D:
I use https://huggingface.co/chat/, which is an open source alternative to ChatGPT.
Their privacy policy is here.
Be aware OP that local LLMs are quite a bit worse than what’s available online. Llama 3 is (probably?) the best one available now and even that has a habit of being very stupid sometimes compared to claude or chatgpt.
I’ve been using GPT4All and quite liking it so far.
Anyone know if there is a way to feed a local AI some of the one notes from my job so I can search like a lazy bastard?
Yeah. You can use ollama, and openwebui. Openwebui allows you to RAG (search your own files) with little effort