#aws-lambda

The Definitive Crash Course on #serverless with #aws: Centralized logging with Kinesis and Lambda
▻https://hackernoon.com/the-definitive-crash-course-on-serverless-with-aws-centralized-logging-w

https://cdn-images-1.medium.com/max/1024/1*OgCtmkQvApUnLEvG05r5gQ.jpeg

Don’t you just hate it when APIs are failing and you have absolutely no clue why? Now imagine you don’t have access to the VM, cluster or container where your software is running. Want me to continue with this nightmare?Yes, that’s what debugging AWS Lambda functions tends to seem like. A horrid nightmare of not knowing what is happening nor why things are failing. This article will show you a way of logging function invocations. Letting you track and monitor failures and errors, while also giving you a nice structure for logging info and debug logs for when you need to troubleshoot behavior.The key is to send all logs to a central location where you can later group, filter and make sense of them. Sematext is a full-stack observability solution for your entire software stack. Meaning you (...)

#monitoring #aws-lambda #programming

Observability Without Having to Break the Bank
▻https://hackernoon.com/observability-without-having-to-break-the-bank-7ebbc861245f?source=rss--

To run a #serverless application in production reliably, you need to be able to debug issues quickly. You need to have good observability, and having structured logs is a big part of that. They allow you to capture useful contextual information with each log message.But observability comes with a cost.The ProblemCloudWatch Logs charges $0.50 for per GB of data ingested, and $0.03 per GB per month for storage. As such, it’s common for companies to spend more on CloudWatch Logs than Lambda invocations in their production environment. Best practices such as using structured logs further exasperate the cost associated with logging.Furthermore, with #aws Lambda, it’s no longer possible to rely on agents to collect and publish observability data such as custom metrics and traces. Instead, you have (...)

#aws-lambda #cloud #cloud-computing

Building a Fully #serverless Realtime CMS using AWS #appsync and #aurora Serverless
▻https://hackernoon.com/building-a-fully-serverless-realtime-cms-using-aws-appsync-and-aurora-se

https://cdn-images-1.medium.com/max/1024/1*9yT_617eObHnSbuYIsDsKw.png

AWS made a serverless option of AWS RDS Aurora generally available, It would mean a breakthrough in building fully serverless architectures with relational database. No more managing connection pools with SQL, No more worrying about RDS instances, Cost is based on the compute time services are consumed, and there is no charge when code is not running.In this post , my goal was to build a fully-functional AWS AppSync-driven backend and #react frontend of a post management app. The app will deliver end users a set of features that allow public users view posts, admin users add/delete post, Let’s start!Application ArchitectureThe application architecture uses:Serverless frameworkAWS AppsyncAWS Cognito Userpool and Identity PoolAWS Aurora ServerlessAWS LambdaPrerequisitesThe following must be (...)

#aws-lambda

Comparing Nuclio and #aws Lambda
▻https://hackernoon.com/comparing-nuclio-and-aws-lambda-14e94c8f1b01?source=rss----3a8144eabfe3-

https://cdn-images-1.medium.com/max/1024/0*YDY_DY3mPnmh_HZv.png

Comparing AWS Lambda and NuclioWith #serverless, you delegate the responsibility of running your infrastructure to a platform provider as much as possible. This frees your engineers to focus on building what your customers want from you — the features that differentiate your business from your competitors’. For this philosophy to work, however, the platform needs to not only give you the tools to build those features, but also deliver the performance that will keep engineers satisfied.Yes, services such as AWS Lambda are powerful, but they have a number of well-documented limitations that make them unsuitable for some workloads. These include:Cold starts and high latency make them unsuitable for real-time applications that require consistent and millisecond-level response times.The (...)

#aws-lambda #aws-lambda-and-nuclio

Making Terraform and #serverless framework work together
▻https://hackernoon.com/making-terraform-and-serverless-framework-work-together-b00e6af63ee9?sou

https://cdn-images-1.medium.com/max/1024/0*YaJLkH9ZMlrnCLMO.png

The Serverless framework is the most popular deployment framework for serverless applications. It gives you a convenient abstraction over CloudFormation and some best practices out-of-the-box:Filters out dev dependencies for Node.js function.Update deployment packages to S3, which lets you work around the default 50MB limit on deployment packages.Enforces a consistent naming convention for functions and APIs.But our serverless applications is not only about Lambda functions. We often have to deal with share resources such as VPCs, SQS queues and RDS databases. For example, you might have a centralised Kinesis stream to capture all applications events in the system. In this case, the stream doesn’t belong to any one project and shouldn’t be tied to their deployment cycles.You still need (...)

#cloud #cloud-computing #aws-lambda #aws

Just how expensive is the full #aws SDK?
▻https://hackernoon.com/just-how-expensive-is-the-full-aws-sdk-abc73d28f62b?source=rss----3a8144

https://cdn-images-1.medium.com/max/1024/0*lZxu_KmnQ4x3oXZk.png

If you’re not familiar with how cold start works within the context of AWS Lambda, then read this post first.When a Node.js Lambda function cold starts, a number of things happen:the Lambda service has to find a server with enough capacity to host the new containerthe new container is initializedthe Node.js runtime is initializedyour handler module is initialized, which includes initializing any global variables and functions you declare outside the handler functionIf you enable active tracing for a Lambda function, you will be able to see how much time is spent on these steps in X-Ray. Unfortunately, the time it takes to initialize the container and the Node.js runtime are not recorded as segments. But you can work out from the difference in durations.Here, Initialization refers to the (...)

#serverless #aws-sdk #aws-sdk-cost #aws-lambda

Getting down and dirty with metric-based alerting for #aws #lambda
▻https://hackernoon.com/getting-down-and-dirty-with-metric-based-alerting-for-aws-lambda-44dee79

https://cdn-images-1.medium.com/max/1024/1*xi_VBKfCHlS816Lr3rfIWA.jpeg

The phrase “better safe than sorry” gets thrown around whenever people talk about monitoring or getting observability into your AWS resources but the truth is that you can’t sit around and wait until a problem arises, you need to proactively look for opportunities to improve your application in order to stay one step ahead of the competition. Setting up alerts that go off whenever a particular event happens is a great way to keep tabs on what’s going on behind the scenes of your #serverless applications and this is exactly what I’d like to tackle in this article.AWS Lambda MetricsAWS Lambda is monitoring functions for you automatically, while it reports metrics through the Amazon CloudWatch. The metrics we speak of consist of total invocations, throttles, duration, error, DLQ errors, etc. You (...)

#aws-lambda #metrics-and-analytics

Deploying a Node.js #twitter Bot on AWS Lambda using #webpack
▻https://hackernoon.com/deploying-a-node-js-twitter-bot-on-aws-lambda-using-webpack-df6e2e187a78

https://cdn-images-1.medium.com/max/1024/1*DieJz9sNLgCBgPlraWy85Q.png

I recently had an idea for a Twitter bot for the #LearnInPublic community, which I thought would be a fun project with half a day worth of work. Moreover, I wanted to use AWS Lambda function for the sake of learning. Why AWS Lambda? Because it’s practically free for maintaining a twitter bot and I got hands on experience with a seemingly complex tool used widely in the industry. You get 1,000,000 free requests and 400,000 seconds of compute time per month for AWS Lambda. My bot uses 0.02% of the above numbers.In the process I found a cool use-case Webpack. I had fun reading the webpack docs and figuring learning about the gotchas. Even though it was a very primitive use-case, felt like a true Sean Larkinn for a day. Here is a detailed article on how you could do the same.A little about (...)

#aws-lambda #nodejs #javascript

Chaos test your Lambda functions with Thundra
▻https://hackernoon.com/chaos-test-your-lambda-functions-with-thundra-48743fe98a91?source=rss---

Failures are inevitable. Just as we need to test our application to find bugs in our business logic before they affect our users. We need to test our application against infrastructure failures. And we need to do it before they happen in production. and cause irreparable damages.The discipline of Chaos Engineering shows us how to use controlled experiments to uncover these weaknesses.In this post, we will see how we can leverage Thundra’s span listeners to inject failures into our #serverless application. We will use these failures to expose weaknesses such as:Missing error handler for DynamoDB operations.Missing fallback when the primary data source is unavailable.Missing timeout on outbound HTTP requests.But first, here’s a quick primer on Thundra.Hello, Thundra!Consider an API with two (...)

#lambda-function #aws #cloud-computing #aws-lambda

#npm Package Verification — Ep. 2
▻https://hackernoon.com/npm-package-verification-ep-2-2b2ec66eb610?source=rss----3a8144eabfe3---

https://cdn-images-1.medium.com/max/300/1*TOWfVGpHDD6Dg4jVUrzf4w.png

NPM Package Verification — Ep. 2Minimum Viable Package Verification as a Service (MVPVaaS)TL;DRI built a super-extra-pre-alpha version of PVaaS (Package Verification as a Service) that is running in the cloud. There is a JSON route that surfaces verification data for a package, and a badge route that, well, gives you a badge.Does a package verify? ▻https://api.verifynpm.com/v0/packages/tbvA previous version? ▻https://api.verifynpm.com/v0/packages/tbv@0.3.0How ‘bout a badge: ▻https://api.verifynpm.com/v0/packages/tbv/badgeI cannot emphasize enough how experimental all of this is! I will not make any breaking changes to the version zero API, however, I can’t make any promises that the API is, ya know, permanent. Feel free to tinker with it. All of the source code is available on GitHub: (...)

#javascript #security #aws-lambda #npm-package-verification

Lambda optimization tip — enable HTTP keep-alive
▻https://hackernoon.com/lambda-optimization-tip-enable-http-keep-alive-6dc503f6f114?source=rss--

https://cdn-images-1.medium.com/max/1024/0*bmImHeGBqn0JWrQz.png

Lambda optimization tip — enable HTTP keep-aliveI recently watched an excellent talk by Matt Levin on optimization tips for Lambda and saw a slide on making DynamoDB use HTTP keep-alive. It reminded me of a conversation I had with Sebastian Cohnen, so I set out to test the effect this simple optimization has.What is it all about?As it turns out, Node.js’s default HTTP agent doesn’t use keep-alive and therefore every request would incur the cost of setting up a new TCP connection. This is clearly inefficient, as you need to perform a three-way handshake to establish a TCP connection. For operations that are short-lived (such as DynamoDB operations, which typically complete within a single digit ms) the latency overhead of establishing the TCP connection might be greater than the operation (...)

#cloud #serverless #aws #cloud-computing #aws-lambda

#analytics services. #serverless analytics
▻https://hackernoon.com/analytics-services-part-3-e3981e694f99?source=rss----3a8144eabfe3---4

Serverless and AWS Lambda how-toThis is the second part of our three-part story about analytical systems. Here you can find Part 1 which answers the questions about client analytics and Part 2 about classical server-side analytics implementation.In this part, we will continue telling you about server-side analytics, particularly about the implementation entirely based on cloud services. In the previous part we have listed the main steps of the implementation. These are:Data queryStream processingDatabaseAggregation serviceFront-endThe same applies here with the only difference in that each step is transferred to cloud services to the greatest extent possible. So, we will follow the main steps listed above in their original order trying to describe potential cloud option properties and (...)

#clickhouse #aws-kinesis #aws-lambda

The Great #serverless Cost Debate(Serverless! = Costless)
▻https://hackernoon.com/the-great-serverless-cost-debate-serverless-costless-1a0b6c07cacb?source

https://cdn-images-1.medium.com/max/1024/1*82UBJ1XUFx7pcfFXivWaEw.jpeg

Source: skitterphoto.comIf you’re worried about switching to a serverless framework being too expensive for your business, you’re not alone. Total spending on cloud services will top $411 billion by 2020. The good news is there are many ways to track and lower the costs of your serverless operation without slowing down your business.So what is AWS Lambda and how can it help your business? Find out more by reading these frequently asked questions.Working with AmazonAWS meaning Amazon Web Services is Amazon’s cloud computing business. AWS serverless is just going into its fifth year in use.Because outsourcing computing power is still so new, it still has many tech workers asking, “What is serverless?” How could using someone else’s servers be efficient or cost effective? Let’s talk about the (...)

#serverless-costless #aws-lambda #analysis #serverless-cost-debate

A crash course on #serverless-side rendering with #react.js, Next.js and #aws Lambda
▻https://hackernoon.com/a-crash-course-on-serverless-side-rendering-with-react-js-next-js-and-aw

https://cdn-images-1.medium.com/max/1024/1*qW57VxADnoD9_o_0L-We0Q.jpeg

Not so long ago I started exploring server-side rendered single-page applications. Yeah, try saying that three times fast. Building products for startups has taught me SEO is a must if you want an online presence. But, you also want the performance SPAs can provide.We want the best of both worlds. The SEO boost server-side rendering provides, and the speed of a Single Page Application. Today I’ll show you all this while hosting it basically for free in a serverless environment on AWS Lambda.TL;DRLet’s run through what this tutorial will cover. You can skim through and jump to the section that interest you. Or, be a nerd and keep reading. whisper Please be a nerd.What’re we building?Configure and install dependenciesBuild the app with the Serverless Framework and Next.jsDeploy the app to (...)

#aws-lambda #nodejs

A crash course on #serverless with #aws — Running Node.js 11 on Lambda
▻https://hackernoon.com/a-crash-course-on-serverless-with-aws-running-node-js-11-on-lambda-6ac67

https://cdn-images-1.medium.com/max/1024/1*PCZ99sB9Pm0Q3SU4CsaSuw.jpeg

A crash course on Serverless with AWS — Running Node.js 11 on LambdaQuite an exciting title, isn’t it? I was hyped when I heard AWS was adding support for custom runtimes and layers for AWS Lambda. This means you can now build your own custom artifacts, enabling you to share and manage common code between functions.I won’t say I fainted when I heard the announcement. But, I did. Don’t tell anybody.What’ll we be doing?This article will show you how to hook up a custom Node.js 11 runtime to AWS Lambda. We’ll create a simple Serverless project with a sample function, and add a layer that will enable us to run the Node.js 11 runtime.How it worksTo use a custom runtime, you have to specify that you’re providing one when deploying your function. When the function is invoked, AWS Lambda will bootstrap (...)

#serverless-with-aws #aws-lambda #nodejs

How to Free #aws Lambda Code Storage (CodeStorageExceeded)
▻https://hackernoon.com/how-to-free-aws-lambda-code-storage-codestorageexceeded-4b7e81c2d575?sou

https://cdn-images-1.medium.com/max/1024/1*ZgBFz4msqZN12zqcEcsMzw.jpeg

You are already up and running with AWS Lambda for several months, and all of a sudden you get the following error:An error occurred: TestDashdeliveryLambdaFunction — Code storage limit exceeded. (Service: AWSLambda; Status Code: 400; Error Code: CodeStorageExceededException; Request ID: 05d3ae68-e7f6–11e8–948e-41c27396380e).What’s the problem?AWS limits the amount of “code storage” it saves on their internal S3 for Lambda functions of up to 75GB.Although it sounds like a lot of space, you can easily reach that limit. In case you’re using the #serverless Framework, its default is to store a version for every deployment you make.Solving the issueIf you don’t need to store a version for each deployment (like many of us), you can easily cancel it with the following addition to your serverless.yml (...)

#lambda-code-storage #aws-lambda-code-storage #aws-lambda

How to Figure Out #aws Pricing for a Mobile Application
▻https://hackernoon.com/how-to-figure-out-aws-pricing-for-a-mobile-application-7fcfc91170ad?sour

https://cdn-images-1.medium.com/max/652/1*GaIekQS6GmpJe95VdZjwsg.png

There are lots of reasons why businesses choose Amazon Web Services cloud for their apps. Companies get storage, analytics, computing, and many other services using one global platform. AWS is taking good care of its clients, lowering tech costs and speeding up the development process. What else can a company owner ask for? You must be thinking about transparent pricing options at the moment.In this article, we’re reviewing AWS features specifically for mobile applications, their pricing, and free trial options. And finally, we’re going to figure out what an AWS calculator is and how it can help you in your work.Benefits of Using AWS for Your Mobile ApplicationDespite the complexity of AWS pricing system, we have to mention its three most important benefits that can help your business (...)

#mobile-app-development #amazon-web-services #aws-lambda

Cons of #serverless Architectures
▻https://hackernoon.com/cons-of-serverless-architectures-7b8b570c19da?source=rss----3a8144eabfe3

https://cdn-images-1.medium.com/max/1024/1*of10JYNtsI0bQQyEoWlxgA.png

A critical look at some of the potential drawbacks of serverless architecture that often get overlooked amidst the hype.Not really but you get the pointThe excitement and fanfare around ‘serverless’ — one of the industry’s favorite buzzwords — continues to grow. Numerous articles, books, and conferences have spun up a hype around serverless talking mostly about the benefits and innovative use cases that serverless enables.Source: Google Trends for Serverless 2016-2018But what about the downsides of Serverless? What are the current limitations that could require complex workarounds, sometimes outweighing the benefits? Amidst all the excitement and the lowered barriers to entry, many are quick to take the plunge without understanding the potential cons that require careful consideration to reap (...)

#cloud #aws-lambda #aws #serverless-architecture

Join the #NoServerNovember Challenge
▻https://hackernoon.com/join-the-noservernovember-challenge-622e36a83e2e?source=rss----3a8144eab

https://cdn-images-1.medium.com/max/1024/1*yritrLHEO_jEPxcpm-NPJw.png

It’s my favorite time of year; the holiday season is in full-swing, we’re preparing for Thanksgiving, and it’s #NoServerNovember. Our team here at #serverless launched #NoServerNovember this year to help serverless enthusiasts up level up and to help new users get started with serverless.Each week in November, we’re announcing new projects for you to try, along with great resources to complete the challenges. Each project you complete and tweet to us using the #NoServerNovember challenge enters you into a drawing to win some Serverless swag. So, try one project this month, tacked a new one each week, or take the title of serverless champion by finishing them all. We’ll be announcing winners all month long with grand prizes announced at the end of the month.The challengesThere are currently 9 (...)

#framework #development #aws-lambda #aws

#aws Lambda with AWS Chalice
▻https://hackernoon.com/aws-lambda-651cb3e17b1c?source=rss----3a8144eabfe3---4

Some notes while working with AWS Lambda using Python3Limit:▻https://docs.aws.amazon.com/lambda/latest/dg/limits.html#limits-listServerless framework by AWS:▻https://github.com/aws/chalicehttps://chalice.readthedocs.io/en/latest/Build extension modules▻https://docs.python.org/2/extending/extending.htmlSome modules will not be compatible with Amazon Linux, they need to be rebuilt for Amazon Linux:- Use build instance: AWS EC2- Use docker: Amazon Linux Container Image: ▻https://docs.aws.amazon.com/AmazonECR/latest/userguide/amazon_linux_container_image.html- Use AWS Cloud9Those extension modules need to be in /vendor of Chalice project:▻http://chalice.readthedocs.io/en/latest/topics/packaging.html#rd-party-packagesAWS Lambda with AWS Chalice was originally published in Hacker Noon on Medium, (...)

#aws-lambda #aws-chalice #serverless #aws-lambda-aws-chalice

Step Functions: how to implement semaphores for state machines
▻https://hackernoon.com/step-functions-how-to-implement-semaphores-for-state-machines-65e23faeee

https://cdn-images-1.medium.com/max/1024/0*v0AMjDokBJYffDSq.png

How do you control the number of concurrent state machine executions that can access a shared resource?Here at DAZN, we are migrat­ing from our lega­cy plat­form into a brave new world of microfron­tends and microser­vices. Along the way, we also dis­cov­ered the delights that #aws Step Func­tion has to offer, for exam­ple…flex­i­ble error han­dling and retrythe under­stat­ed abil­i­ty to wait between tasksthe abil­i­ty to mix auto­mat­ed steps with activ­i­ties that require human interven­tionIn some cas­es, we need to con­trol the num­ber of con­cur­rent state machine exe­cu­tions that can access a shared resource. This might be a busi­ness require­ment. Or it could be due to scal­a­bil­i­ty con­cerns for the shared resource. It might also be a result of the design of our state machine which makes it dif­fi­cult to (...)

#cloud #aws-lambda #cloud-computing #serverless

How to deploy a Node.js application to #aws Lambda using #serverless
▻https://hackernoon.com/how-to-deploy-a-node-js-application-to-aws-lambda-using-serverless-ae7e7

https://cdn-images-1.medium.com/max/1024/1*ERi-zAqQ8X1D7Q2xDyVmyw.jpeg

Being a developer is amazing. Writing code, solving problems and thinking of ingenious solutions for complicated algorithms is what we live for. But, the grass is not always so green on this side of the fence. Sooner or later you need to get your hands dirty and deploy the app you worked so hard on. Deployments are not always easy. To be blunt, they can be incredibly hard and time-consuming. That’s what we’ll solve in this tutorial.TL;DRYou can follow along from the beginning or jump to the section which interests you most, and severely hurt my feelings. ?GoalsPrerequisitesWhat’s AWS Lambda?But, how does it work?What will we deploy?How to deploy it?Let’s get coding!Ready to deploy!Deploying to production!How to gain insight into your system?GoalsToday you’ll learn how to deploy a Node.js (...)

#web-development #aws-lambda #nodejs

How to create IP-protected endpoints with API Gateway and Lambda
▻https://hackernoon.com/how-to-create-ip-protected-endpoints-with-api-gateway-and-lambda-41db007

If you haven’t been pay­ing close atten­tion you might have missed the API Gate­way announce­ment for resource poli­cies. It lat­er played a key role in sup­port­ing API Gate­way pri­vate end­points — a way to put your API inside a pri­vate VPC.To con­fig­ure resource poli­cies with the Server­less frame­work, you need to upgrade to v1.28.0 or lat­er. If you want to restrict access to the GET /index.html end­point to the IP 217.128.123.174, you need the fol­low­ing.provider: name: #aws runtime: nodejs8.10 resourcePolicy: - Effect: Allow Principal: "" Action: execute-api:Invoke Resource: - execute-api://GET/index.html Condition: IpAddress: aws:SourceIp: - 217.128.123.174Nice and easy!There are a cou­ple of things to note:You can (...)

#aws-lambda #cloud #cloud-computing #serverless

Centralised logging for #aws Lambda, REVISED (2018)
▻https://hackernoon.com/centralised-logging-for-aws-lambda-revised-2018-e88f22aabb0d?source=rss-

https://cdn-images-1.medium.com/max/1024/0*zis4dbJRplUAKiqN.png

First of all, I would like to thank all of you for fol­low­ing and read­ing my con­tent. My post on cen­tralised log­ging for AWS Lamb­da has been viewed more than 20K times by now, so it is clear­ly a chal­lenge that many of you have run into.In the post, I out­lined an approach of using a Lamb­da func­tion to ship all your Lamb­da logs from #cloud­Watch Logs to a log aggre­ga­tion ser­vice such as Logz.io.In the demo project, I also includ­ed func­tions to:auto-sub­scribe new log groups to the log-ship­ping func­tionauto-update the reten­tion pol­i­cy of new log groups to X num­ber of days (default is Nev­er Expire which has a long term cost impact)This approach works well when you start out. How­ev­er, you can run into some seri­ous prob­lems at scale.Mind the concurrencyWhen pro­cess­ing Cloud­Watch Logs with (...)

#serverless #cloud-computing #aws-lambda

Five ways to gain #serverless #observability
▻https://hackernoon.com/five-ways-to-gain-serverless-observability-406e5cab51af?source=rss----3a

https://cdn-images-1.medium.com/max/567/1*fNZUlBcLh5qm36mfU6Na-Q.png

In this post, I’m covering five approaches for gaining observability in serverless (FaaS based) systems. I hope that it can help developers, system architects, and DevOps engineers make the serverless journey successful, less frustrating, and of course — enjoyable.Serverless and Function-as-a-Service (FaaS) have been around for about three years now. It started with AWS Lambda, and quickly expanded into similar services provided by the leading cloud vendors — Microsoft, Google, IBM, and others. However, as people started using it, serverless became much more than just FaaS. Serverless systems today involve functions, containers, managed services provided by the cloud vendor (e.g., message queues, DBs, storage), and a vast variety of SaaS APIs (e.g., Auth0, Twilio, Stripe), all interacting (...)

#distributed-systems #aws-lambda #monitoring