Solved

Unable to write object to Amazon S3: The request signature we calculated does not match the signature you provided. Check your key and signing method.

  • 11 October 2021
  • 1 reply
  • 5623 views

Userlevel 6
Badge +3
  • Senior Technical Evangelist and Developer Advocate at Snowflake
  • 67 replies

I am trying to write to Amazon S3 destination with its Authentication Method set to AWS Keys, but when I run the pipeline I get “Unable to write object to Amazon S3: The request signature we calculated does not match the signature you provided. Check your key and signing method.” error.

Here’s the entire stack trace…

Caused by: com.streamsets.pipeline.api.StageException: S3_21 - Unable to write object to Amazon S3, reason : com.amazonaws.services.s3.model.AmazonS3Exception: The request signature we calculated does not match the signature you provided. Check your key and signing method. (Service: Amazon S3; Status Code: 403; Error Code: SignatureDoesNotMatch; Request ID: 12345678915ABCDE; S3 Extended Request ID: xyzxyzxyzxyzxyzxyz=; Proxy: null), S3 Extended Request ID: xyzxyzxyzxyzxyzxyz=
at com.streamsets.pipeline.stage.destination.s3.AmazonS3Target.write(AmazonS3Target.java:182)
at com.streamsets.pipeline.api.base.configurablestage.DTarget.write(DTarget.java:34)
at com.streamsets.datacollector.runner.StageRuntime.lambda$execute$2(StageRuntime.java:303)
at com.streamsets.datacollector.runner.StageRuntime.execute(StageRuntime.java:244)
at com.streamsets.datacollector.runner.StageRuntime.execute(StageRuntime.java:311)
at com.streamsets.datacollector.runner.StagePipe.process(StagePipe.java:221)
at com.streamsets.datacollector.runner.preview.PreviewPipelineRunner.lambda$runSourceLessBatch$0(PreviewPipelineRunner.java:364)
at com.streamsets.datacollector.runner.PipeRunner.acceptConsumer(PipeRunner.java:207)
at com.streamsets.datacollector.runner.PipeRunner.forEachInternal(PipeRunner.java:152)
at com.streamsets.datacollector.runner.PipeRunner.executeBatch(PipeRunner.java:132)
at com.streamsets.datacollector.runner.preview.PreviewPipelineRunner.runSourceLessBatch(PreviewPipelineRunner.java:360)
at com.streamsets.datacollector.runner.preview.PreviewPipelineRunner.processBatch(PreviewPipelineRunner.java:284)
at com.streamsets.datacollector.runner.StageRuntime$3.run(StageRuntime.java:383)
at java.security.AccessController.doPrivileged(Native Method)
at com.streamsets.datacollector.runner.StageRuntime.processBatch(StageRuntime.java:379)
at com.streamsets.datacollector.runner.StageContext.processBatch(StageContext.java:299)
at com.streamsets.pipeline.lib.jdbc.multithread.JdbcBaseRunnable.handlePostBatchAsNeeded(JdbcBaseRunnable.java:421)
at com.streamsets.pipeline.lib.jdbc.multithread.JdbcBaseRunnable.generateBatchAndCommitOffset(JdbcBaseRunnable.java:314)
at com.streamsets.pipeline.lib.jdbc.multithread.JdbcBaseRunnable.run(JdbcBaseRunnable.java:178)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:226)
at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:34)
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:222)
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeRunnable.run(SafeScheduledExecutorService.java:188)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:226)
at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:34)
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:222)
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeRunnable.run(SafeScheduledExecutorService.java:188)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
... 3 more

 How do I resolve this issue?

icon

Best answer by Dash 11 October 2021, 23:08

View original

1 reply

Userlevel 6
Badge +3

It turns out that there's a known issue with AWS credentials that contain certain characters. For example, +, % and /. So in order to get past this error you should regenerate your access keys and try again. For more details on this, refer to this thread.

Reply