Ciao, yes for now the sentiment analysis is used only to provide an emoji related to the response. In the future I would like also to influence the choices of adjectives and interjections according to the sentiment.
This is such a nice idea! I could add a fallback towards LLMs, it was present but I removed it. Would you be interested to help me implement the auto-update? I must admit, the LLMs are very useful for this kind of work. I think that TERMY's design is now feasible BECAUSE OF the availability of LLMs. They make the dataset development feasible.
Yes tldr, navi cheatsheets are essentially a great set of how to do X using Y for terminal utilities. Pretty well curated set of commands for majority of the tasks, so are great training material.
And great work! Love it
I haven't evaluated it yet, but I love the fact that the output is (at least claimed to be) deterministic. I can't trust an LLM to do the right thing after I deploy it to production, because their output is non-deterministic by design.
TERMy (or is it the NPC-forge) seems to be worth a try.
You can get determinostic output (mostly) by setting the temperature to zero. Using couple of other tricks you can get close to 100% of determinism with LLMs.
That's reproducible, I wouldn't call it deterministic. Small, semantically meaningless changes in the input can still result in wildly different output.
That's the definition of a chaotic system (small change in initial conditions results in large, seemingly -- but not actually -- random changes in output), but it's still deterministic (same input results in same output).
> because their output is non-deterministic by design.
It isn't. At least not by design, even though in practice it often can be. If you do greedy decoding (or use a preset seed) and deterministically compute everything (e.g. only use integer math) then it will be 100% always deterministic.
That’s true, but not true-true. Sure, every time you prompt “what is the weather in kansas” you’ll get the same output, but if you prompt “what is the weather in kansas right now” you’ll get a different output, and then “what is the weather in kansas today” gets a different output. Language being language, there are infinite ways to say things, so there are infinite variations in what the llm can output in response to very similar prompts.
This tool has a finite amount of outputs for an infinite amount of inputs. Which is different from an llm based tool.
Yes i think I acknowledged that, but is that useful for making a tool that can be trusted to safely run shell commands when asked arbitrary questions? No. It’s not.
Cool, but system and user should probably stick to short, clear commands. E.g., I see you do some anaphora resolution (in particular: find what "it" refers to), but in a complex dialog, the human intention can differ from the machine's understanding. That will give problems when you end your dialog with "delete it".
Adding more sentences to your data set will slowly degrade performance. It's a delicate system.
Source: I have written software with similar functionality (NLP search) in SaaS form, a long time ago. It required quite a bit of work to configure.
> Models like ornith:9b, mistral:7b or cogito:14b can get the job done sometimes, but they are not fast and reliable enough for general use, specially if you have only 4GB of VRAM.
Have you considered/tried using a model that's, well, more appropriate size-wise for an use case like this? These are relatively big. Something like FunctionGemma [1] finetuned for a given set of tasks would be a lot more speedy.
FunctionGemma never worked well for me (without fine tuning). Liquid has released 230M and 350M models that work far, far better in my testing: https://huggingface.co/LiquidAI/LFM2.5-230M
I really look forward to a hypothetical LFM3-230M, because LFM2.5-230M is so close to being usable, while FunctionGemma is miles away from being usable.
I tried functiongemma, it is for sure faster than those models, the problem is that is not reliable enough for a terminal assistant. I would say that no LLM is good for a terminal assistant, if you take into account the operational cost and the risk of damage. Even if it fails only 1 time out of 10 becomes useless. That's why I developed FlintParser!
Warp uses LLMs so it is slow and prone to hallucination. Using very colloquial terms TERMy is more or less a calculator that knows english :) so it can run on your CPU and respond instantly! The difference is that it can only answer predetermined responses (with optional arguments) this makes it useless if you need to generate text, but makes it safe and predictable for a use case like a terminal assistant.
is it supported to have it propose a command for approval rather than running autmatically? in the YT video it looks likw it ran the cpu temp command on its own
Yep he runs on its own when the command is non-destructive, like checking the CPU temperature, it does ask for permission if the command is potentially destructive. I agree it is so cool, it looks sci-fi :)
To be clear, the word "human" is used determine who can use the software because I am convinced that, when a company, institution, organization or corporation is created, it has its own intelligence, means and interests that go beyond and above the interests of human beings.
Many are scared of the singularity, thinking that general artificial intelligence out of human control is the greatest threat for humanity.
I am convinced that the greatest threat for humanity is more probably just a corporation operating to achieve its own interests.
A one man company consulting another would apply the software for commercial purposes. Commercial use is not comtemplated by the license. The one man company should contact me and buy a different license.
I am not sure about:
>"This is my project, you can use it if you want. If you want a patch added, pay me. If you want support, pay me."
Because if you are an experimenter asking for a patch I could be also interested in, that could help the project in any way, I could consider doing it for free as part of effort required to develop the project.
The idea is, let experimenters tinker with it and help you develop your idea, while at the same time exclude companies, institutions, organizations and corporations. If the software is interesting for them they can contact the copyright holder to buy a different license.
Instead of giving away for free our work to entities that may work against our own interests, we give it away for free only to people like us.
Hobby clubs could not use it because they are an organization, members of the club could use it because they are individuals playing with their hobby. Non-profit organizations could not use it.
I must say often non-profit organizations even if not producing money have strong conflicts of interest.
> It needs to supports, out of the box, cloud connected or wifi enabled (wireless lan connected) devices.
When the physical layer of devices in the home don’t offer Wireless connectivity, hardware communication modules needs to be adopted to bridge the existing connectivity.
For now it is a bit of a gimmick I agree :)
reply