youtube-upload runs from terminal but not cron call

Question

the following is my up427.sh script being called from cron under ubuntu 12.06 on digitalocean server droplet.

the script runs and uploads to youtube from the terminal on that droplet with bash up427.sh but not when bash up427.sh is done from crontab on that droplet

it echos a error 126 to the error out file i have made called cx.txt

I know that means 126 Command invoked cannot execute /dev/null Permission problem or command is not an executable

keep in mind it does run when i run

bash up427.sh

from the droplet terminal and it works and uploads to youtube so the sh command and syntax are correct from terminal when running the script from terminal using bash

i am root.

my crontab is the universal one created by me as root using crontab -e so I beleive it has all priviledges.

the following is the call from crontab to up427.sh

SHELL=/bin/bash
HOME=/root/
#PATH=/sbin:/bin:/usr/sbin:/usr/bin:/root
MAILTO=”me@mail.com”
#PATH=/usr/local/share/youtube_upload:/root:/sbin:/bin:/usr/sbin:/usr/bin
#Uses client secrets at: /usr/local/share/youtube_upload/client_secrets.json
#Uses credentials fileat : /root/.youtube-upload-credentials.json



57 * * * *  bash /root/up427.sh; echo $?; echo$*; echo $$ > /root/up427dirfromcron.txt

(obviously i adjust the 57 to whatever minute i want it to run)

the script does a cd /root from the beginning which is where all these files and the related subdirectoris are located.

additionally i have made the crontab try both home=/ anmd hpme=/root/ and it didnt make a difference

all files are called by their specific subdirectory to avoid cron job subdirectory vageries (which happenend until I did that)

I have also tried it with and without all the sh headers. no difference.

begin actual verbatim script file up427.sh (thedate i began ripping out my hair at 3am)

SHELL=/bin/bash
HOME=/root
#PATH=/sbin:/bin:/usr/sbin:/usr/bin:/root
#MAILTO=”me@mail.com”
PATH=/usr/local/share/youtube_upload:/root:/sbin:/bin:/usr/sbin:/usr/bin


bash2run="youtube-upload --client-secrets=/usr/local/share/youtube_upload/client_secrets.json --credentials-file=/root/.youtube-upload-credentials.json 
--title=mytitle --privacy=private /root/thefileiwanttouploadwhichisfineandworksfromcommandline.mp4 "


echo $bash2run  >/root/cx.txt


youtube-upload --client-secrets=/usr/local/share/youtube_upload/client_secrets.json --credentials-file=/root/.youtube-upload-credentials.json 
--title=mytitle --privacy=private /root/thefileiwanttouploadwhichisfineandworksfromcommandline.mp4

echo $?  >>/root/cx.txt  (this echoes the error 126 when runby cron)

echo $*  >> /root/cx.txt  

echo $$  >> /root/cx.txt

echo "---------end of up427.sh-------------------"  >>/root/cx.txt

end of the verbatim sh file up427.sh

again the cx.txt file simply holds the errors out from running the sh file under crontab so I can see what is not happening.

I have tried to call youtube-upload in this sh file using all permeatations of directories. none work when this sh is called from the cron

there is a subdirectory under /root/youtube-upload
I dont even see the program in there which is odd.

the program tales all those following arguments in the quoted variable.

please note i have the variable there just to export it to the cx.txt error out log file but i actually call the program using bash as you can see in the next line.

i have tried (in the script ) bash "you-upload --arguments" with no qoutes. doublke quotes. single quotes etc. in the sh file.

I have included everything i can think of in thepath. doesnt seem to make a difference.

the program comes from

https://github.com/tokland/youtube-upload

i have looked at many crontab answers here but none apply and make you-upload work from the cronjob schedule!

yes the program DOES get called.

the error out in cx.txt shows that.

my dir structure is

/home/root/
then under that
drwxr-xr-x  7 root root     4096 Apr 24 16:04 google-api-php-client
drwxr-xr-x  3 root root     4096 Apr 24 17:40 .subversion
drwxr-xr-x  3 root root     4096 Apr 24 17:40 install
drwxr-xr-x  6 root root     4096 Apr 24 17:46 youtube-upload
-rw-r--r--  1 root root    17833 Apr 24 17:47 master.zip.1
drwxr-xr-x  6 root root     4096 Apr 24 17:47 youtube-upload-master
 -rw-r--r-- 1 root root   855 Apr 28 12:36 up427.sh

addtionally the directory of /youtube-upload is

root@(none):~/youtube-upload# ls -l -t -r -a 
total 52
-rw-r--r--  1 root root  1247 Apr 24 17:46 setup.py
-rw-r--r--  1 root root  4920 Apr 24 17:46 README.md
-rw-r--r--  1 root root   725 Apr 24 17:46 .gitignore
drwxr-xr-x  2 root root  4096 Apr 24 17:46 examples
-rw-r--r--  1 root root   522 Apr 24 17:46 client_secrets.json
drwxr-xr-x  2 root root  4096 Apr 24 17:46 bin
drwxr-xr-x  6 root root  4096 Apr 24 17:46 .
drwxr-xr-x  3 root root  4096 Apr 24 17:46 youtube_upload
drwxr-xr-x  8 root root  4096 Apr 24 17:46 .git
drwx------ 11 root root 12288 Apr 28 01:28 ..

please offer suggestions

I dont know if this is a error with the github project file scope or cron scope or some stupid typo

when i run it from the temrinal using

bash up427.sh

this is the successfull output

 root@(none):~# bash  up427man.sh
 Using client secrets: /usr/local/share/youtube_upload/client_secrets.json
 Using credentials file: /root/.youtube-upload-credentials.json
 Start upload: /root/thefileiwanttouploadwhichisfineandworksfromcommandline.mp4
 Video URL: https://www.youtube.com/watch?v=(theyoutubevideonumber)
 (randomcodespecifictoyoutubeasuthwhichiwontshowhere)

some people asked for the following

echo $SHELL
/bin/bash
root@(none):~# ps -p $$
 PID TTY          TIME CMD
20144 pts/1    00:00:00 bash
root@(none):~# which bash
/bin/bash

somemore thoughts

I am beginning to think it may be having trouble parsing the command for youtube-upload -arguments -arguments-arguments

I am thinking parts need to be in quotes when called by cron but not when the bash sh file is done from terminal. just a guess. could be totally wrong. trying many variations of quotes around parts now.

I don't know anything about the headers at the top of sh or crontab files. like the bin/bash thing and then misisng ! etc.

could that be important? what should be atthe top of both the crontab -e edited file

and my up427.sh file?

and the wierd thing is i can't for the life of me figure out what the name of the program actually is that i run from the bash or the command line that runs with

youtube-upload

because that seems to be a directory name and I can't actually find the programmed named that yet it runs! could it be some environment variable that invokes another program when typed?

when I start adding directory preface to it such as /root/youtube-upload

I get things like "program 'youtube-upload' is a directory"

yet the invokcation is

youtube-upload --title=blahblah -- decription=blahblah

as it shows on the git page!

in its purest working form running from the command line in the form

bash up427.sh

up427.sh is

youtube-upload --client-secrets=/usr/local/share/youtube_upload/client_secrets.json --credentials-file=/root/.youtube-upload-credentials.json --title='titleonyoutube' --privacy=private /root/filetobeuploaded.mp4

thats without all the nonesense that sends txt to a file to show me outputs when running under a cronjob.

I hope that helps.

that works when run as

bash up427.sh

from the temrinal commandline

however it returns a error 127 when called from the crontab file

update

after trying to use the EVAL statement in the script that was called from crontab

this is the new script with multiple variations of the call to the youtube-upload program in many quotation mark variations etc to see if any would work. each variation in calling was named a different upload name on Youtube name so i could see which got uploaded to youtube.

just as before none worked when called from cron

so eval didn't help.

however as before numerous variouations uploaded when using both the new eval and the old echo statements when the same script was called from the terminal as before

these are the variations of the call that got uploaded when the script was called from the terminal.

as you can see

four using eval worked

and four using the old method of echo worked

2avartest2 HD

2commbashworksvartest2 HD

avartest2 HD

commbashworksvartest2 HD

2avartest2eval HD

2commbashworksvartest2eval HD

avartest2eval HD

commbashworksvartest2eval HD

once againthey only worked when the script was run from the terminal and not when run from cron!

ARRGGHHHHH DOUBLE TRIPLE ARRRRGGHHHH!

here is the new script with many (8) instances using eval and the same old 8 using echo which didnt work either

(I can't figureout how to paste the whole thing in as a huge codeblock so it isnt indented becuase I can't indent all these lines manually. But frankly its easier to read withthe wrap around anyway.)

    commbashworks="youtube-upload --title=commbashworksvartest2eval --client-secrets=/usr/local/share/youtube_upload/client_secrets.json --credentials-file=/root/.youtube-upload-credentials.json --privacy=private /root/uptest.mp4"

eval $commbashworks

echo

a='youtube-upload --title=avartest2eval --client-secrets=/usr/local/share/youtube_upload/client_secrets.json --credentials-file=/root/.youtube-upload-credentials.json --privacy=private /root/uptest.mp4'

eval $a

echo

b="/root/youtube-upload --title=bvartest2eval --client-secrets=/usr/local/share/youtube_upload/client_secrets.json --credentials-file=/root/.youtube-upload-credentials.json --privacy=private /root/uptest.mp4"

eval $b

echo

c='/root/youtube-upload --title=cvartest2eval --client-secrets=/usr/local/share/youtube_upload/client_secrets.json --credentials-file=/root/.youtube-upload-credentials.json --privacy=private /root/uptest.mp4'

eval $c

echo

d="../root/youtube-upload --title=dvartest2eval --client-secrets=/usr/local/share/youtube_upload/client_secrets.json --credentials-file=/root/.youtube-upload-credentials.json --privacy=private /root/uptest.mp4"

eval $d

echo

e='../root/youtube-upload --title=evartest2eval --client-secrets=/usr/local/share/youtube_upload/client_secrets.json --credentials-file=/root/.youtube-upload-credentials.json --privacy=private /root/uptest.mp4'

eval $e

echo

#

commbashworks="youtube-upload --title=2commbashworksvartest2eval --client-secrets=/usr/local/share/youtube_upload/client_secrets.json --credentials-file=/root/.youtube-upload-credentials.json --privacy=private uptest.mp4"

eval $commbashworks

echo

a='youtube-upload --title=2avartest2eval --client-secrets=/usr/local/share/youtube_upload/client_secrets.json --credentials-file=/root/.youtube-upload-credentials.json --privacy=private uptest.mp4'

eval $a

echo

b="/root/youtube-upload --title=2bvartest2eval --client-secrets=/usr/local/share/youtube_upload/client_secrets.json --credentials-file=/root/.youtube-upload-credentials.json --privacy=private uptest.mp4"

eval $b

echo

c='/root/youtube-upload --title=2cvartest2eval --client-secrets=/usr/local/share/youtube_upload/client_secrets.json --credentials-file=/root/.youtube-upload-credentials.json --privacy=private uptest.mp4'

eval $c

echo

d="../root/youtube-upload --title=2dvartest2eval --client-secrets=/usr/local/share/youtube_upload/client_secrets.json --credentials-file=/root/.youtube-upload-credentials.json --privacy=private uptest.mp4"

eval $d

echo

e='../root/youtube-upload --title=2evartest2eval --client-secrets=/usr/local/share/youtube_upload/client_secrets.json --credentials-file=/root/.youtube-upload-credentials.json --privacy=private /uptest.mp4'

eval $e

echo

#

commbashworks="youtube-upload --title=commbashworksvartest2 --client-secrets=/usr/local/share/youtube_upload/client_secrets.json --credentials-file=/root/.youtube-upload-credentials.json --privacy=private /root/uptest.mp4"

don't replacethisone

echo $($commbashworks)

echo

a='youtube-upload --title=avartest2 --client-secrets=/usr/local/share/youtube_upload/client_secrets.json --credentials-file=/root/.youtube-upload-credentials.json --privacy=private /root/uptest.mp4'

echo $($a)

echo

b="/root/youtube-upload --title=bvartest2 --client-secrets=/usr/local/share/youtube_upload/client_secrets.json --credentials-file=/root/.youtube-upload-credentials.json --privacy=private /root/uptest.mp4"

echo $($b)

echo

c='/root/youtube-upload --title=cvartest2 --client-secrets=/usr/local/share/youtube_upload/client_secrets.json --credentials-file=/root/.youtube-upload-credentials.json --privacy=private /root/uptest.mp4'

echo $($c)

echo

d="../root/youtube-upload --title=dvartest2 --client-secrets=/usr/local/share/youtube_upload/client_secrets.json --credentials-file=/root/.youtube-upload-credentials.json --privacy=private /root/uptest.mp4"

echo $($d)

echo

e='../root/youtube-upload --title=evartest2 --client-secrets=/usr/local/share/youtube_upload/client_secrets.json --credentials-file=/root/.youtube-upload-credentials.json --privacy=private /root/uptest.mp4'

echo $($e)

echo

#

commbashworks="youtube-upload --title=2commbashworksvartest2 --client-secrets=/usr/local/share/youtube_upload/client_secrets.json --credentials-file=/root/.youtube-upload-credentials.json --privacy=private uptest.mp4"

echo $($commbashworks)

echo

a='youtube-upload --title=2avartest2 --client-secrets=/usr/local/share/youtube_upload/client_secrets.json --credentials-file=/root/.youtube-upload-credentials.json --privacy=private uptest.mp4'

echo $($a)

echo

b="/root/youtube-upload --title=2bvartest2 --client-secrets=/usr/local/share/youtube_upload/client_secrets.json --credentials-file=/root/.youtube-upload-credentials.json --privacy=private uptest.mp4"

echo $($b)

echo

c='/root/youtube-upload --title=2cvartest2 --client-secrets=/usr/local/share/youtube_upload/client_secrets.json --credentials-file=/root/.youtube-upload-credentials.json --privacy=private uptest.mp4'

echo $($c)

echo

d="../root/youtube-upload --title=2dvartest2 --client-secrets=/usr/local/share/youtube_upload/client_secrets.json --credentials-file=/root/.youtube-upload-credentials.json --privacy=private uptest.mp4"

echo $($d)

echo

e='../root/youtube-upload --title=2evartest2 --client-secrets=/usr/local/share/youtube_upload/client_secrets.json --credentials-file=/root/.youtube-upload-credentials.json --privacy=private /uptest.mp4'

echo $($e)

echo

#

UPDATE SOLUTION FOUND!

correction it didnt work . it deleted files.

Answer 1

My first guess would be that your default shell is not Bash and that when the script is executed under cron it is be run under a different shell. The reasons I guessing this is that you are experiencing some difference from CLI to CRON is:

1. I noticed that when run interactively, you are specifying bash via bash up427.sh; and
2. you don't specify the shell with at the top of your script.

To help us troubleshoot:

• Can you add your crontab entry to your post?
• Can you tell us what you see when you run echo $SHELL at the terminal?

UPDATE

Based on your updates, it looks like a mismatched shell maybe not be the root cause. Given the changing error code (from 126 to 127) there has to be more involved. To completely rule out the mismatched shell, can you try the following from the terminal of the target server (logged into the terminal as the same user the crontab is running under) and report back the output:

$ echo $SHELL
$ ps -p $$
$ which bash

Update #2
I would recommend the use of eval. To do so, you would build up up the full command as a string that would be passed to eval. This would be used instead of echo $bash2run >/root/cx.txt. A basic example of the idea is:

my_args='-al'
eval "ls ${my_args}"

If you work up from this basic example, I think you will get it to work.

For small but informative discussion on the use of eval, exec and source, I recommend you read this thread on bash shell: 'exec', 'eval', 'source' - looking for help to understand

In addition, I would recommend running the script in debug mode by adding the -x flag like such:

bash -x up427.sh

Answer 2

UPDATE SOLUTION FOUND! well the solutionin fact deleted some random files so I have taken it off until I figure out why.

I leave this important note from before

ANYTIME YOU USE CRON I cannot emphasize enough the importance of specifying completely the full path of all sh files, files in arguments and any files you want to pipe output too such as >> /mydirectorywhereiwantit/output.txt in full in both your crontab file and your sh files script files.

As I learned cron will look for files and write files in whatever directory it happens to be in while bashing and that can get EXTREMELY confusing if that is a different directory from where you normally type from the terminal. SO FULLY SPECIFY ALL PATHS TO ALL FILES IN BOTH CRONTAB AND IN THE CALLED SH SCRIPT FILES AND IN ARGUMENTS!

Thanks to John Mark Mitchell who so kindly offer so many possible solutions.