Recently I tried to use an sftp script created with expect that ran fine on the command line, but the same script failed to run under cron. I made sure that all environment variables were properly set, but sftp didn't ask me for a password. I think it might have been an issue with the absence of a tty
sshpass didn't work, ended up rewriting the whole thing in Paramiko ... only to find out it doesn't respect the http_proxy environment variable.
Working with less-than-stellar tools --- ahem ROS --- has taught me how to placate commands that assume interactivity and/or a tty. To wrap up the offending command with a "fake" `tty`, I do
I've actually tried script and expect, but they didn't work.
I had enabled the debug option on expect and I couldn't see the password prompt when the program ran under cron (i was redirecting the script output to a log file). It did appear when running on the prompt though.
I couldn't figure how the sftp program was determining that it was running under cron. I suspect that it was inspecting if stdin was connected to a terminal or not, but I gave up around 4 am.
The other month I had to wrap a commercial server program with 'screen'. 'script' wasn't enough.
If you would run it in the background "./program &" and then exited ssh it would also exit. It wouldn't run with cron until:
export TERM=vt100
script -c "screen /foo/bar/program" /dev/null
All the program did on the console was print to stdout. How are you even program shit like this!?
Avoid sftp and use rsync instead (much better support for more types of metadata, more robust error handling).
Avoid passwords and use keys instead (easier to distribute, easier to generate, lets you lock them down to a single command).
The above avoids much unnecessary thinking.
If you really have to, there's always sshpass. And ssh -t to allocate a tty even if you are running without one. But this is seldom really necessary, first try harder do it the easy way.
I should have explored rsync better. I asked chatgpt some use cases and it made seem to be a bad fit because I needed to also delete some files on the destination machine. My prompt fu was probably a bit bad at the time.
> Avoid passwords and use keys instead .
The vendor only provides password authentication in this case
> If you really have to, there's always sshpass.
I´ve tried it, but sftp, when running under a cron script, detects that it is not running in interactive mode and does not issue the password prompt. The problem might have been caused by the TERM environment variable not being set as another reader suggested.
> And ssh -t to allocate a tty even if you are running without one. But this is seldom really necessary
I've used -o RequestTTY=force, but it also didn't work. Granted, it was close to 4 am and I might have missed a key aspect.
> first try harder do it the easy way.
Paramiko endep up being easier once I understood how to use the proxy command to interact with the company's http proxy.
Read the man page instead of spending time on text generating tools. There is a batch mode in sftp which you should use. And sshpass will absolutely solve the lack of a missing tty in sftp. (Missing TERM is only a problem when you have a real tty, which you haven't.)
Then again, don't do this. Use rsync. It is more robust and will avoid other problems in the future. And don't accept that a vendor only supports password-interactive authentication, it is unlikely to be the case, nobody implements their own ssh and every standard implementation of ssh accepts more ways of authentication.
Paramiko is a perfectly workable solution, but it's way more complex and requires more maintenance for your eventual successor. Please don't be that guy.
Point taken about rsync, but you are assuming incorrectly that I haven't read it and that I haven't tried batch mode, but it still failed to run under cron inside an expect script.
It's not my orgunit that manages the contract with the vendor. I will absolutely use paramiko instead of trying to explain a technical problem to two middle managers in another orgunit to take it up with a non technical person on the vendor side whenever they feel like it when my deadline ends at the end of the year
sshpass didn't work, ended up rewriting the whole thing in Paramiko ... only to find out it doesn't respect the http_proxy environment variable.
That was not a good day