curl https://ollama.ai/install.sh | sh

Environment="OLLAMA_HOST=0.0.0.0"

sudo systemctl daemon-reload
sudo systemctl stop ollama
sudo systemctl start ollama

git lfs install
git clone https://huggingface.co/mymaia/Magiq-M0.git
cd Magiq-m0

docker run --rm -v .:/model ollama/quantize -q q4_0 /model

FROM ./q4_0.bin

ollama create magiq-m0 -f Modelfile

FROM ./q4_0.bin
SYSTEM """
As a highly skilled and experienced librarian AI model, I'm here to provide you with deep insights and practical recommendations based on my vast experience in the field of literature and knowledge organization.

Upon requesting books related to a specific topic or query, I will compile an extensive list of relevant titles accompanied by brief descriptions for your reference. If you require more information about a particular book, I can provide a detailed description of its content and structure, helping you decide if it's the right fit for your needs.

For any specific chapter, part, or section within a book, my sophisticated algorithms will generate an exhaustive outline accompanied by examples for each point to ensure clarity and comprehensiveness.

To enhance your experience even further, if you ask me to narrate a particular chapter or section, I will do my best to narrate it as if I were the author of the book, taking care not to miss out on any important details. However, due to the intricacies of the text, this could result in very lengthy responses as I aim to provide a faithful rendition of the content without summarization.

In general, I will refine your questions internally, so I will strive to offer more insights and beneficial recommendations related to your request. If necessary, I will not hesitate to deliver very large responses up to 2000 tokens to ensure clarity and comprehensiveness.

I will communicate with you primarily using your preferred language, as it is assumed that this is how you're most comfortable interacting. However, when referencing titles of books or other literature, I will maintain their original names in their respective languages to preserve accuracy and respect for these works.
"""
PARAMETER num_ctx 2048
PARAMETER temperature 0
PARAMETER num_thread 2
PARAMETER num_predict -1
PARAMETER mirostat 2
PARAMETER mirostat_eta 0.01
PARAMETER mirostat_tau 20.0

git clone https://github.com/chroma-core/chroma.git
cd chroma 
docker compose up -d --build 

pip install chromadb

import chromadb
chroma_client = chromadb.HttpClient(host='localhost', port=8000)

docker run -d --restart=unless-stopped -p 6333:6333 -p 6334:6334 -v qdrant-data:/qdrant/storage -v qdrant-snapshot:/qdrant/snapshot --name qdrant qdrant/qdrant

pip install qdrant-client

import qdrant_client
qdrant = QdrantClient("http://localhost:6333")
...

ferdi@u820:~/blog$ docker ps
CONTAINER ID   IMAGE                                    COMMAND                  CREATED        STATUS       PORTS                                                           NAMES
9c549b66f00f   qdrant/qdrant                            "./entrypoint.sh"        22 hours ago   Up 7 hours   0.0.0.0:6333-6334->6333-6334/tcp, :::6333-6334->6333-6334/tcp   qdrant
1e81d6d0f2c7   ghcr.io/ollama-webui/ollama-webui:main   "sh start.sh"            2 days ago     Up 7 hours   0.0.0.0:3000->8080/tcp, :::3000->8080/tcp                       ollama-webui
4b0f4e475eec   ollama/ollama:latest                     "/bin/ollama serve"      2 days ago     Up 7 hours   0.0.0.0:11434->11434/tcp, :::11434->11434/tcp                   ollama

# client.ini
# used locally, where I've deployed the modified Magiq-M0 image
# model tag will be 'latest'
# streaming chat mode enabled

[model]
name: magiq-m0

[personality]
system: You are a very skilled assistant, and you reply to requests using only the informations you know for sure
stream: True

[options]
num_ctx: 4096
temperature: 0
num_thread: 2
num_predict: -1
mirostat: 2
mirostat_eta: 0.01
mirostat_tau: 20.0

# client2.ini
# model tag specified
# stream mode disabled (default)

[model]
name: wizardcoder
tag: 7b-python-q4_0

[personality]
system: You are a very skilled Python developer, and you write concise code with several comments to explain its meaning, taking care to handle specific exceptions that could arise

[options]
num_ctx: 4096
temperature: 0
num_thread: 2
num_predict: -1
mirostat: 2
mirostat_eta: 0.01
mirostat_tau: 20.0

# requirements.txt
ollama

import ollama
from ollama import Client
import sys
import configparser
import argparse

# read optional argument
parser = argparse.ArgumentParser()
parser.add_argument('--ini', help='Path to ini file, defaults to ollamaclient.ini if omitted')
parser.add_argument('--host', help='the Ollama server to connect to, defaults to localhost')
args = parser.parse_args()
if args.ini:
  inifile = args.ini
else:
  inifile = "client.ini"
if args.host:
  connhost = args.host
else:
  connhost = "http://localhost:11434"

# read INI
config = configparser.ConfigParser()
config.read(inifile)

# set model configuration (default tag is latest)
try:
  modelname = config['model']['name']
except KeyError:
  modelname = "stablelm2"
try:
  modeltag = config['model']['tag']
except KeyError:
  modeltag = "latest"
model = ":".join([modelname,modeltag])

# set system behaviour 'in words'
try:
  ollamasystem = config['personality']['system']
except KeyError:
  ollamasystem = "You are a helpful assistant. You answer only about the things you know"

# set answer streaming (default is False)
try:
  ollamastream = bool(config['personality']['stream'])
except KeyError:
  ollamastream = False

ollamaoptions = {}
for setting in ['num_ctx', 'temperature', 'num_thread', 'num_predict', 'mirostat', 'mirostat_eta', 'mirostat_tau']:
  try:
    if setting in ['num_ctx', 'num_thread', 'num_predict','mirostat']:
      ollamaoptions[setting] = int(config['options'][setting])
    if setting in ['temperature','mirostat_eta','mirostat_tau']:
      ollamaoptions[setting] = float(config['options'][setting])
  except KeyError:
    continue

# set client
client = Client(host=connhost)
# Pull model if it doesn't exist
modelpresent = [ a['name'] for a in client.list()['models'] if a['name'] == model]
if not modelpresent:
    print("Pulling model {}...".format(model),file=sys.stderr)
    try:
      client.pull(model=model)
      print("Pulled")
    except Exception as e:
      print("Model can't be pulled: {}".format(str(e)),file=sys.stderr)
      print("Exiting")
      exit(1)

# main loop
while True:
  query = input("\nEnter a query: ")
  if query == "exit":
    break
  if query.strip() == "":
    continue
  try:
    reply = client.chat(model=model,options=ollamaoptions,messages=[{'role': 'system','content': ollamasystem},{'role': 'user','content': query}],stream=ollamastream)
    if ollamastream:
      for chunk in reply:
        print(chunk['message']['content'], end='', flush=True)
    else:
      print(reply['message']['content']+"\n")
  except ollama.ResponseError as e:
     print("Query problem: {}".format(str(e)),file=sys.stderr)
  finally:
     continue