The whole site looks like and reads like AI slop. The outcomes also don't make any sense and don't feel rigorously tested (no, having claude test for you doesn't count as rigorous).
Please stop making this comment. The war is lost. Instead, you should be commenting that it looks like a human wrote this when you come across the rare brain-produced writing
I'm the same way, I have a very low/sporadic usage of any subscription I've tried. I now just use openrouter with DS4 pro/flash. It also gets rid of usage anxiety where I would try to justify the $20/month by forcing myself to use the tokens for projects as the weekly limit deadline neared.
The way I think about it is that it's unreasonable for a compute graph with a static number of operations to be able to answer both y=a*10 and something like y=((((x+x)*(x+1))/((2*x)+2))+((x*(x+3))/(x+3))-((x*x)/(x+1))+((x*x)/(x+1))-((x*(x+3))/(x+3))) in a single forward pass. Tokens are essentially a unit of work and can also be used for intermediate steps, not just final results.
I'm not a heavy user of agentic coding, but still use them quite a bit for some automation here and there. I've been going around shopping all the ~$10 subscriptions and I finally settled on openrouter + ds4 pro. The more intensive days cost me $1 and I set a $2 weekly limit which I've never hit the past 3 weeks, to me it's way cheaper than most subscriptions and I don't have to worry about maximizing my weekly quota/resets.
The posted answers are either behind a paywall or very obtuse so I'll just explain. I'll assume you know what tokens are.
A models output is not a single token, but a list with the probability for all the tokens that it knows, so we need to use a sampler to select the token that it's going to be the next token in the sentence. For example a simple greedy sampler will choose the token with the highest probability, but samplers normally pick a random token weighted by probability. A model usually knows about ~250 thousand tokens and the probability of some of these tokens are gonna be high, but the vast majority is close to but not actually 0% so there's a chance the sampler might pick some random token that doesn't make much sense, so we filter tokens.
top_k filters the tokens so that only the k top tokens are selected. So top_k=50 will filter those 250k tokens to only 50. This is assuming the list of tokens is sorted by probability.
top_p filters the top tokens until a percentage is accumulated. So if for example if you set the top_p to 0.6 and the model gave the top token a 0.5 (50%) probability and the second top token a 0.2, those 2 token accumulated to 0.7 which is greater than what you set it to (0.6) so no more tokens are selected. If this ran after top_k=50 it'll turn the list of 50 tokens into one of 2.
After each filter parameter is processed, the probability of the tokens is adjusted to sum to 1 (100%), Also note that order of operation here matters, i.e. top_p could be applied before top_k, but most providers follow what's on huggingface, I think I've only seen different implementation in certain local model hosting frameworks.
I used to try to train models ages ago (that is, a couple of years), and I never found out how they got good results from top_k in the first place.
My problem was that as soon as the model had generated an unlikely token, it was quickly steering towards spaces it had never seen during its training. It found itself in unmapped territory, where anything might happen. With high temperatures, it would then generate noisy garbage, with low temperature it would generate repetitive garbage. We tried to fight this with rollout strategies and backtracking, i.e. if you find yourself in a space where all tokens seem equally likely, you're probably in junk land.
Our tricks didn't work very well. But I didn't manage to keep up well enough to learn what worked.
> So they are essentially a protection against spurious errors
Only coincidentally. Sampling nonsense tokens will certainly degrade its performance and/or brick it, but it's also there to encourage diversity.
For example, imagine we have the following sentence:
> The color of this ball is ____
Now, what should the model predict for "____"? There isn't really a "correct" answer here. It can be "red", it can be "blue", it can be "green", or any other color. But it's definitely not going to be "ব্যথাя". LLMs output a probability distribution for the next token, so imagine this is the probability distribution that it outputs:
red -> 60%
green -> 19%
blue -> 19%
ব্যথাя -> 2%
So how do we decide which token to pick? Simplest way is to always pick the most probable one (in this case: "red"). In this case we'd ideally want it to be able to output "red", "green" or "blue" (since all of those are reasonable), but never "ব্যথাя" (whose 2% is most certainly noise). So a sampler is essentially an algorithm which lets the inference engine pick the exact token to output from this list.
> I don't quite understand the point about order of operations - does it do normalization after every such filter pass? why not leave it to the end?
Because you can technically compose multiple samplers at the same time in a pipeline, and in some cases their order can matter and give you a different result (or take less/more time to execute). To give you a generic example: imagine you have a list with numbers in random order. You can execute one of two operations on it: (1) sort it, (2) take the leading 10 numbers. If you first sort it and then take 10 leading numbers you'll get a different result than if you'd first took 10 leading numbers and then sorted them.
Yes, the whole list will always sum to 1 (100%) because there's lots of more sampling parameters. top_p, top_k and temperature are just the ones that affect output the most. Most parameters do math around assuming the list sums to 1 and order is not always the same, some software even lets you change the order around.
top_a is not very common and is better explained if I explain how the much more common min_p works. min_p filters out tokens below a certain threshold. The formula is <filter threshold> = <min_p> * <top token probability>. So if the top token has 0.5 probability, min_p = 0.1 would cut out tokens below 0.05. This is a tunable that lets you filter out other tokens depending on how confident the model is.
top_a is almost the same formula but you just square the <top token probability>. So <filter threshold> = <top_a> * <top token probability> ^ 2. This makes the filtering ramp up faster (cut out more tokens) if the model has a much more confident top choice, but keep more choices if the model is not so confident.
Yup, it's the main reason I don't use LM studio more. I only use it to try out new models/quants, then use llama.cpp directly to host them. LM Studio also doesn't do stuff like audio input and often has bugs that pure llama.cpp doesn't so it can be a net negative for certain use cases.
It doesn't matter until it does. If the chinese government decides that open weight model releases are no longer allowed, that's a lot of companies that can't release new models. Same with the US government, etc. Having diversity is important.
However unlike the US models, China banning the release of new models would not break existing ones. Betting on US models only can get you locked out in just a few hours.
No, Open weights US models would not break as well - this isn't related to China or USA, it's about Open Weights and the fact that you can download the models.
It's a similar problem the human DNA solved by telling our teenage selves that our parents are dumb and we needed to move to a new tribe. Genetic diversity, but a digital equivalent.
This hasn't been tested in court. But there's a high chance that model weights are not copyrightable, only the code to generate them is.
Cloud models are usually protected by trade secret laws, leaking them would get you in trouble. However if the model is made available publicly, as long as you don't break the law to get them, anything after that would be fair game unless Apple can prove that humans have significant authorship over the weights, which hasn't been tested and is a significant burden to prove/disprove.
Yes. Having 4 names are quite common in Portugal, specially in certain areas. The names are usually structured like this: G1 G2 FM FF
G1 and G2 are given names. Usually 2 "first names" that you see in english, but there's common combos and sometimes there's a word joining them. Examples: "Maria Jesus" vs "Maria de Jesus". Some names are more common to be put first, but almost every name can be put in any order, example: "José António" vs "António José".
FM and FF are easy. FF is the family name of your father (your father's FF), and FM is the family name from your mother (your mother's FF).
Where I was raised 99% of my friends had 4 names structured like this, I only knew a few that didn't. When I moved to Lisbon the 3 name structure was much more common, dropping the second given name.
In Portugal there's rules for naming your kids (at least there were when I lived there), but I think in Brazil such rules don't exist. The author is brazillian but his name seems to follow the traditional portuguese naming style, as you guessed his name in english could be translated to "Robert Anthony Smith of Almeida" (Almeida is a portuguese town).
FF is your "first last name" and FM is your "second last name".
FF is your father first last name (his FF).
And FM is your mothers first last name (hers FF).
The FF FM order was how it used to be (at least in Chile). Now, when the first kid of a given couple is born, that couple choose the order (FF FM or FM FF). In any case, is always the first last name of the parents, and the chosen order must be used for all kids in common between them.
The losers are quickly forgotten. Palm, Blackberry, AOL, MySpace. Yahoo, etc.
Software gets replaced all the time too, you even listed one and didn't realize. 15 years ago you'd call office irreplaceable, now you have to add gsuite to the mix, in 15 years there might be others. I know people that have never had office installed on their PC and use spreadsheets daily.
> It seems that enterprises will pay top dollar for service guarantees, integration, and someone they can sue.
Of course. But why pay $25 per million tokens for sonnet when you can pay $3 for GLM? Both probably running on AWS/Azure/Etc. under some third party.
Individuals, sure. For enterprise you can't get monthly plans. You have to pay per token.
It's a bit like saying "nobody pays for Microsoft Office". I certainly don't know anyone personally who has. Students get a free Education License and then your employer provides one for you...
Strange. This is what they got at my company. I do not remember anyone mentioning paying for tokens. Maybe because it is fairly small, couple of hundred people in IT.
Officially it's not available anymore. But there are team plans that are not enterprise, so if you are small enough and fine with the data protection included in those maybe that is what you're working with? And I do know NGOs were offered seat based plans after they were officially not available anymore.
Also: This change came in in March so if you got your contract before then this will only bite once you renew.
Ah past renewal... So probably in next contract there will be a lot of thinking if we still need that.
Curious: Anybody have comparison what is the difference when company is changing to token based billing? How many times it is bigger than 200$ subscription?
