Accessing local SQS service from another docker container using environment variables

Question

I have a flask application which needs to interact with an SQS service whenever an endpoint is hit. I'm mimicing the SQS service locally using docker image sukumarporeddy/sqs:fp whose base image is https://github.com/vsouza/docker-SQS-local with two more queues added in configuration.

I need to access this service from another app which is run as app_service. These two services are run using docker-compose.yml file where I mentioned two services.

app_service

sqs_service

While building the app image, I'm setting environment variables to access the sqs_service as QUEUE_ENDPOINT=http://sqs_service:9324. But when I try to access the sqs_service the app, it is saying invalid queue endpoint.

I'm using boto3 to connect to the local sqs_service.

boto3.client('sqs', endpoint_url=os.getenv("QUEUE_ENDPOINT"), region_name='default')

Here's the docker-compose.yml file.

  app_service:
    container_name: app_container
    restart: always
    image: app
    build: 
      context: ./dsdp
      dockerfile: Dockerfile.app.local
    ports:
      - "5000:5000"
    env_file:
        - ./local_secrets.env
    command: flask run --host=0.0.0.0 --port 5000

  sqs_service:
    container_name: sqs_container
    image: sukumarporeddy/sqs:fp
    ports:
      - "9324:9324"

local_secrets.env:

QUEUE_ENDPOINT=https://sqs_service:9324
FEEDER_QUEUE_URL=https://sqs_service:9324/queue/feeder
PREDICTION_QUEUE_URL=https://sqs_service:9324/queue/prediction
AWS_ACCESS_KEY_ID=''
AWS_SECRET_ACCESS_KEY=''

Error that I'm getting when trying to send messages to the SQS service that is running locally.

ValueError

ValueError: Invalid endpoint: https://sqs_service:9324

Where am I making mistake?

Answer 1

I think there is no configuration for SSL in your docker-compose file, so the issue with might be with https. try to change https to http.

QUEUE_ENDPOINT=http://sqs_service:9324
FEEDER_QUEUE_URL=http://sqs_service:9324/queue/feeder
PREDICTION_QUEUE_URL=http://sqs_service:9324/queue/prediction

Also, try to call SQS container by name without using ENV for debugging.

import boto3
sqs=boto3.client('sqs', endpoint_url="http://sqs_service:9324", region_name='default')
# Create a SQS queue
response = sqs.create_queue(
    QueueName='SQS_QUEUE_NAME',
    Attributes={
        'DelaySeconds': '60',
        'MessageRetentionPeriod': '86400'
    }
)

print(response['QueueUrl'])

Tested with alpine-sqs image.

Answer 2

Somehow I was not able to use the SQS queue using the service name mentioned in the environment variables. This is what I did instead.

Got the service name from environment variables, got the IP address of the service using socket library in python and used the IP address to format and create the QUEUE endpoint url.

import socket
queue_endpoint_service = os.getenv("QUEUE_ENDPOINT_SERVICE")
queue_endpoint_port = os.getenv("QUEUE_ENDPOINT_PORT")
feeder_queue = os.getenv("FEEDER_QUEUE")
prediction_queue = os.getenv("PREDICTION_QUEUE")
queue_endpoint_ip = socket.gethostbyname(queue_endpoint_service)
queue_endpoint = f"http://{queue_endpoint_ip}:{queue_endpoint_port}"
mc = boto3.client('sqs', endpoint_url=queue_endpoint, region_name='default')
feeder_queue_url = f"{queue_endpoint}/queue/{feeder_queue}"
prediction_queue_url = f"{queue_endpoint}/queue/{prediction_queue}"

I'm able to send the messages now by hitting the endpoints in flask app.

Note: Also used the new docker image which Adiii mentioned. Not using the old image anymore.