I have a batch file C:\upload_to_s3.bat. In this file, there is a line:
aws s3 sync D:\S3\batch1\ s3://MyBucket/batch1 --exclude *.bat
I have Windows task scheduler "S3 Hourly Sync" that runs every hour to trigger to run C:\upload_to_s3.bat. But this command does not do anything - the file upload never happened.
It runs perfectly if I double click on C:\upload_to_s3.bat.
This is Windows 2008 Standard server. I have installed AWS CLI and configured with the command "aws configure", and entered my access key and secret key. That is why it runs if I double click the batch file.
The Windows Task Scheduler is run as "[MyServer]/Administrator" account.
So why doesn't the "aws s3 sync" command work when triggered by the task scheduler?
Do I need to add a line before "aws s3 sync" to set credentials first? If so, how?
Thank you!
i was able to solve the problem by adding the following 3 lines in the batch file BEFORE your s3 command
aws configure set AWS_ACCESS_KEY_ID <your acess key id> aws configure set AWS_SECRET_ACCESS_KEY <your secret access key here> aws configure set default.region eu-west-1
replace the stuff between angled brackets with your data and the region with your bucket's region. and yes ,there is no equal sign between "AWS_ACCESS_KEY_ID" and the key.
Include the following lines in your batch file:
set AWS_ACCESS_KEY_ID=your_access_key_id
set AWS_SECRET_ACCESS_KEY=your_secret_access_key
AWS Documentation
The reason this is happening, is because aws command is not finding the the config or the credentials file! And it happens for Win2012 too.
The problem is that Windows Task Schedule sets the %USERPROFILE% to C:\Users\Default instead of the actual user the task is running under!! ??? And AWS is using that to find the config and credentials files.
So the fix is to put something like this in your script (either BAT or PowerShell)
:: in a BAT script set userprofile=C:\Users\%username% # in a PowerShell $env:USERPROFILE = "c:\users\$env:USERNAME"
Even the %homepath% variable is blank when running from a Scheduled Task.
I just discovered this after procrastinating trying to figure out why this was happening for us.
set AWS_DEFAULT_PROFILE=namedprofile set AWS_CONFIG_FILE=C:\Users\xxxx\.aws\config
Only until I added the AWS_CONFIG_FILE line above in my script was AWS able to actually see the config file! However it can not find the credentials now, and I don't see anywhere how to set that.
This is the error I get without telling where the config file is:
Traceback (most recent call last): File "aws", line 27, in <module> File "aws", line 23, in main File "awscli\clidriver.pyc", line 50, in main File "awscli\clidriver.pyc", line 176, in main File "awscli\clidriver.pyc", line 157, in _create_parser File "awscli\clidriver.pyc", line 91, in _get_command_table File "awscli\clidriver.pyc", line 111, in _build_command_table File "botocore\session.pyc", line 680, in emit File "botocore\hooks.pyc", line 226, in emit File "botocore\hooks.pyc", line 209, in _emit File "awscli\customizations\preview.pyc", line 71, in mark_as_preview File "awscli\clidriver.pyc", line 349, in service_model File "awscli\clidriver.pyc", line 366, in _get_service_model File "botocore\session.pyc", line 256, in get_config_variable File "botocore\session.pyc", line 277, in _found_in_config_file File "botocore\session.pyc", line 349, in get_scoped_config botocore.exceptions.ProfileNotFound: The config profile (backup) could not be found
And then this is the error I get after setting the AWS_CONFIG_FILE variable:
Unable to locate credentials Completed 1 part(s) with ... file(s) remaining
In windows bat file what works is a combination of user2415376 and docesam answers.
Here it is:
set userprofile=C:\Users\%username% aws configure set AWS_ACCESS_KEY_ID <your_key> aws configure set AWS_SECRET_ACCESS_KEY <your_secret> aws configure set default.region <your_region> other aws commands here
