I deployed my React website build/ folder into an AWS S3 bucket.
If I go to www.mywebsite.com, it works and if I click on a ta
Why it happens
Your issue is that you want to pass responsibility to routing to your react app/javascript. The link will work because react can listen to the link click and simply update the route in the browser URL bar. However, if you go a location where your script (index.html and the bundle.js or wherever your app code is located) is not loaded, then the JavaScript is never loaded and has no chance to handle the request. Instead, whatever runs your server will take care of the request, look if there is a resource at this place, and return a 404 error or whatever else it found.
The solution
As mentioned in the comments, this is the reason why you need to redirect 404-errors to the exact location where your app is placed. This is nothing specific to Amazon, it is a general requirement to set up your react app.
To solve the issue you need to find out what handles routing on your server and how to configure it. For example if you have an Apache server, an .htaccess file could take care of the matter like this:
RewriteEngine On
RewriteBase /
RewriteRule ^index\.html$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.html [L]
This file lets the server redirect not found errors to the index.html. Keep in mind that it might affect other routing rules that are on your server, so this configuration is easiest, if your react app has a place on its own, not interfering with other stuff.