Reddit » Amazon Web Services
589 FOLLOWERS
A subreddit for news, articles and tools covering Amazon Web Services (AWS), including S3, EC2, SQS, RDS, DynamoDB, IAM, CloudFormation, Route 53, CloudFront, Lambda, VPC, Cloudwatch and Glacier.
Reddit » Amazon Web Services
21h ago
I really appreciate anyone being willing to help. Just to refresh, this is the blog post I've been following: This is the link to the blog I have been following along with: https://aws.amazon.com/blogs/machine-learning/intelligently-split-multi-form-document-packages-with-amazon-textract-and-amazon-comprehend/ -
and the github - https://github.com/aws-samples/aws-document-classifier-and-splitter/
So since my original post I was able to get the state machine to actually work and stop looping, but only when I significantly lowered the number of multi-page pdf's I was giving it from S3 in order ..read more
Reddit » Amazon Web Services
21h ago
Hey all,
I am trying to follow this process for a one time issue I am solving.
Cloudwatch logs -> S3 -> Glue -> S3
Where the glue job will pick and tranform some data and place it in the next S3 bucket
I have a Grok pattern to classify the data is and plan to run the glue job on that, however the s3 export puts an empty line between the logs I am exporting
ie:
``` 2023-12-21T04:33:11.209Z {"level":"INFO", "test":"true"}
2023-12-21T05:33:11.209Z {"level":"INFO", "test":"false"}
2023-12-21T06:33:11.209Z more test text
2023-12-21T07:33:11.209Z {"level":"INFO", "test":"yes"}
2023-12-21T0 ..read more
Reddit » Amazon Web Services
21h ago
How do you customize the tags karpenter uses to identify the security groups and subnets it needs. I do not want to use karpenter.sh/discovery,Value=${CLUSTER_NAME} rather I would like to use my own custom key value pair. I already have an existing cluster, sg, subnets, roles, etc.
Any examples, tutorials of the appropriate yaml would be helpful
submitted by /u/No-Replacement-3501
[visit reddit] [comments ..read more
Reddit » Amazon Web Services
21h ago
I gave the loop interview month back and received the following email after 5 days . Congratulations, following your Virtual On-Site Loop interview, I'm pleased to share you that the outcome appears to be very positive. The feedback from the team indicates a strong inclination towards moving forward. Currently, the team is in the process of discussing the next steps. I want to assure you that we are committed to keeping the momentum going. In the coming days, @email, will be reaching out to you directly to walk you through the offer process and address any questions or concerns you may have ..read more
Reddit » Amazon Web Services
21h ago
What options are there to scan and verify our windows servers have the minimal files/software installed on them? I saw that system manager has a document called " AWS-ApplyDSCMofs " but it seems kind of old. Is there something else that would be able to achieve compliance. Example, verify server has certain .txt files and anti-virus installed.
submitted by /u/newbie702
[visit reddit] [comments ..read more
Reddit » Amazon Web Services
21h ago
Hello everyone!
I was hoping I could get some expert advice from you fine folks.
Long story short, I'm very new to data engineering and I have the following project.
Each morning ~6am a very large file is dropped into an S3 bucket. I want to transform this file and output it into another S3 for analytics queries using Athena. This file is tricky to work with as it is:
Compressed with gzip
Often >30gb in size whilst compressed
Not in csv format, it's effectively log lines
So far I have had some success using AWS Glue notebooks as a scratch pad with pyspark, managing to apply to a dynamic ..read more
Reddit » Amazon Web Services
21h ago
Hi All,
I have been tasked with finding the unused EBS disk space total per account. Is there a way to find the unused and used ebs volume space? Does AWS hide this value since this is a you pay for what you provision for?
The AWS CLI does not appear to have fields for Used space and unused space.
I have been playing with boto3 and looking at the documentation, but unsure if there is a way to pull this as well.
submitted by /u/anapayuhbray
[visit reddit] [comments ..read more
Reddit » Amazon Web Services
21h ago
Good afternoon guys, could you ask me a question? How do I force a user to enable MFA?
submitted by /u/kgrgc
[visit reddit] [comments ..read more
Reddit » Amazon Web Services
21h ago
So I am very new to AWS and I am trying to deploy my project which is a Docker container, via AWS.
I already have AmazonECS_FullAccess and the Admin policy permissions for my IAM user, and created a very basic Express app POC that includes a health route, and which is Dockerized (which works perfectly on localhost), and then pushed to AWS ECR successfully, and the image successfully uploaded. I even went ahead and created a new ECS cluster and a new task successfully, where I enabled the health check option. Now first when I created a service, it kept on failing due to the circuit breaker.
I ..read more
Reddit » Amazon Web Services
21h ago
Hey all, my friend wrote this awesome post on how to properly cache docker layers for github actions using AWS ECR. Give it a read!
https://blacksmith.sh/blog/push-cache-repeat-amazon-ecr-as-a-remote-docker-cache-for-github-actions
submitted by /u/drakefromcanada
[visit reddit] [comments ..read more