The connection you are drawing is not incorrect in my opinion. But perhaps I can clarify.
The dynamics model in a KF is a linear system with an input and and output (often denoted u and y respectively).
A Hidden Markov Model typically does not have an input. There is an initial state, and then the system transitions "autonomously" (that's a term I'm borrowing from differential equations).
An "Markov model" with input is called a Markov Decision Process (MDP). And if the state of the system is not fully observable, then it's a POMDP.
So Kalman filters are most analogous to "belief updates" in POMDPs. The KF takes into account the known input to the system.
The dynamics model in a KF is a linear system with an input and and output (often denoted u and y respectively).
A Hidden Markov Model typically does not have an input. There is an initial state, and then the system transitions "autonomously" (that's a term I'm borrowing from differential equations).
An "Markov model" with input is called a Markov Decision Process (MDP). And if the state of the system is not fully observable, then it's a POMDP.
So Kalman filters are most analogous to "belief updates" in POMDPs. The KF takes into account the known input to the system.