AWS provides great services and can be easily configured to handle big traffic spikes. However, sometimes some nuance occurs when a big amount of data need to process. For example, I was using SQS in my Laravel project. We are letting the Credential Provider retrieve the credentials to access SQS service via the IAM role attached to the EC2 instance. This works great but due to the volume of traffic we get this exception thousands of times a day:

Error retrieving credentials from the instance profile metadata server. (cURL error 28: Connection timed out after 1000 milliseconds (see https://curl.haxx.se/libcurl/c/libcurl-errors.html))

Well… Seems I have a challenge again 🙂

First of all, to understand how this works I can explain it in a few works. When code in your EC2 instance tries to use other AWS services (for example SQS, S3) it should have permission to do this. These permissions can be configured using AWS IAM roles. To get the information, what EC2 instance can do with other services, the request to the AWS metadata server will be done with each of your code request to another service. If EC2 will do a lot of requests to the instance metadata service (IMDS) then results in the calls being throttled, timeout errors will occur.

After some exploring, I found that AWS-SDK has a function that allows memoizing the credentials. But this was not enough. The problem with this is that it only stores the credentials for the script execution, meaning the next script ran will fetch the credentials again from the meta server.

Cache AWS Credentials in a EC2 Instance

Solution? Ideally, we need a solution that caches these credentials into a file so we only go to the meta server when the current credentials are due to expire. Why do not cache it for example to Redis? Because it will be one more connection to other services to get credentials information.

The simple way to cache your credentials can be done like this:

use Aws\Sqs\SqsClient;
use Aws\Credentials\CredentialProvider;
use Aws\DoctrineCacheAdapter;

$params = [
'version' => 'latest',
'region' => 'us-east-1',
'credentials' => new DoctrineCacheAdapter(new FilesystemCache('/tmp/cache'))
 ];
$sqsClient = new SqsClient($params);

Hope this will help!