#cloud-computing

Deconstructing #serverless Computing Part 1: A new layer of abstraction
▻https://hackernoon.com/deconstructing-serverless-computing-part-1-a-new-layer-of-abstraction-98

“All problems in computer science can be solved by another level of indirection.” — David WheelerIn a series of articles, Lucian Toader explores serverless computing, looking at it from a business perspective as well as from a developer’s point of view.We have come a long way since the 50s; mainframes and time-sharing were replaced by virtual machines and cloud computing, those in turn giving way to containers, with the creation of Docker. Each step until now has added a new layer of abstraction, simplifying provisioning and server management through increased automation, while bringing down costs by exploiting the commoditization of hardware, software and economies of scale.IaaS, PaaS, SaaS, and the missing linkAs information and communications technology (ICT) evolves, we are gaining (...)

#serverless-computing #functions-as-a-service #cloud-computing #function-as-a-service

#google Cloud Next ’19
▻https://hackernoon.com/google-cloud-next-19-4fbedaff95e7?source=rss----3a8144eabfe3---4

https://cdn-images-1.medium.com/max/1024/1*uNhlavSa9u6d9j7NQWnmVQ.png

Last week the tech world gathered at the Moscone Center in San Francisco for Google Cloud Next, the annual event focussing on Google’s latest cloud solutions, the future of AI and machine learning. A lot of what was announced has an impact on our work. Let’s take a look at the exciting news!Google Cloud versus the restThis was the first edition Google Cloud Next with Thomas Kurian as CEO of Google Cloud, as he took over from Diane Green end of last year. He made it clear that Google will be focusing more on enterprise — while still embracing startups — and open source to strengthen its multi-cloud infrastructure offering.We’re committed to substantially expanding the scale of our go-to-market teams in order to help more customers use our technologyThomas Kurian, CEO Google CloudAs we can see (...)

#cloud-computing #google-cloud-platform #google-next #public-cloud

What I learned by migrating an #rds #database into production using #aws DMS
▻https://hackernoon.com/what-i-learned-by-migrating-an-rds-database-into-production-using-aws-dm

https://cdn-images-1.medium.com/max/1024/1*x488oKVrrB7H3d1NB37VvQ.jpeg

What I learned by migrating an RDS database into production using AWS DMS (Database Migration Service)…or how to avoid a big headache while live migrating and replicating your db instancesImage from Jan Antonin Kolar — UnsplashI am currently managing the migration of some on premise services, and other services already present on AWS under a single VPC.Having active production systems H24 7/7 it was essential to design and implement a fairly precise migration plan with a zero error margin.While regarding services like EC2 instances, EFS, Security Group, Target Group and Load Balancer the transition was quite simple, I encountered some obstacles during the migration of our RDS database (MariaDB) from one VPC to another.For this task the native Amazon service, DMS (Database Migration Service) (...)

#cloud #cloud-computing

Observability Without Having to Break the Bank
▻https://hackernoon.com/observability-without-having-to-break-the-bank-7ebbc861245f?source=rss--

To run a #serverless application in production reliably, you need to be able to debug issues quickly. You need to have good observability, and having structured logs is a big part of that. They allow you to capture useful contextual information with each log message.But observability comes with a cost.The ProblemCloudWatch Logs charges $0.50 for per GB of data ingested, and $0.03 per GB per month for storage. As such, it’s common for companies to spend more on CloudWatch Logs than Lambda invocations in their production environment. Best practices such as using structured logs further exasperate the cost associated with logging.Furthermore, with #aws Lambda, it’s no longer possible to rely on agents to collect and publish observability data such as custom metrics and traces. Instead, you have (...)

#aws-lambda #cloud #cloud-computing

A Beginners Guide to Federated Learning
▻https://hackernoon.com/a-beginners-guide-to-federated-learning-b29e29ba65cf?source=rss----3a814

https://cdn-images-1.medium.com/max/698/1*l-Y_3nqlR-eNCRQSDXV-vg.jpeg

We predict growth of Federated Learning, a new framework for Artificial Intelligence (AI) model development that is distributed over millions of mobile devices. Federated Learning models are hyper personalized for an user, involve minimum latencies, low infra overheads and are privacy preserved by design. This article is a beginner level primer for Federated Learning.Disclaimer: the author is an investor and advisor in the Federated Learning startup S20.ai. In case you are wondering, S20 stands for “Software 2.0”.The AI market is dominated by tech giants such as #google, Amazon and Microsoft, offering cloud-based AI solutions and APIs. In the traditional AI methods sensitive user data are sent to the servers where models are trained.Recently we are seeing the beginning of a decentralized (...)

#federated-learning #artificial-intelligence #machine-learning #cloud-computing

Things No-one Tells You About #cloud #migration Projects
▻https://hackernoon.com/things-no-one-tells-you-about-cloud-migration-projects-b7c2e4b880a3?sour

A list of lessons learned from the thrills and spills of various cloud migration projects.“This was an environment built, not for man, but for man’s absence.” J.G BallardI’ve been in the fortunate position to have been involved in several cloud migration projects in my career. Here’s a few insights I’ve collected which should hopefully help others starting out on their own cloud migration journey.It’s hard work.This work isn’t easy. There can be a surprising number of challenges to overcome, and certainly not just Technical challenges, for example:Organisational Politics — e.g. stakeholders in the project may lose their confidence in the team if it takes a long protracted time for delivery of the work without any tangible results in the interim.Resourcing — e.g. some individuals may be better (...)

#cloud-computing #software #devops

The Future of Cloud Computing with Joshua Strebel from Pagely
▻https://hackernoon.com/the-future-of-cloud-computing-with-joshua-strebel-from-pagely-e502435cd4

https://cdn-images-1.medium.com/max/1024/1*yX-RVj0IjB3M-u2P-esoNQ.png

Episode 36 of the Hacker Noon Podcast: An interview with Joshua Strebel, CEO of Pagely and NorthStack.▻https://medium.com/media/0d1b47d6c24b20039a870c9dfbb22208/hrefListen to the interview on iTunes, or Google Podcast, or watch on YouTube.In this episode Trent Lapinski and Joshua Strebel discuss #serverless, cloud computing, #devops, #wordpress, and Joshua’s new project NorthStack.“You talk about AI and specifically around content publishing, there’s those crazy algorithms now that you can give it a sentence and a closing and it’ll write a thousand words in between and it will be on point. You’re like ‘I couldn’t have written this any better!” — Joshua StrebelThe Future of Cloud Computing with Joshua Strebel from PagelyProduction and music by Derek Bernard — haberdasherband.com/productionHost: Trent (...)

#hackernoon-podcast #cloud-computing

Cloud cost hacking
▻https://hackernoon.com/cloud-cost-hacking-fc35fd19985d?source=rss----3a8144eabfe3---4

https://cdn-images-1.medium.com/max/960/1*BrrM25rh_NXrVDk7k90zPA.jpeg

One of the primary motivations for using the cloud are flexibility and cost saving. The problem is that without ongoing management and oversight, the cloud budget can grow to dimensions that eliminate profitability.A recent study on the cost of using the cloud has shown that companies pay an average of 35% more on cloud services than they really should. A total of over $60 billion is being wasted on cloud services that are not in use.In this article, I will share my conclusions from recent cloud cost optimization processes and review a number of ways and tools to significantly reduce cloud costs 30–50%, without sacrificing performance. The recommendations in this article are valid for all cloud vendors unless otherwise specified.Tools for managing cloud costsBefore you start saving it is (...)

#devops #google-cloud-platform #cloud-computing #azure #aws

The Rise of Zero-Trust #architecture
▻https://hackernoon.com/the-rise-of-zero-trust-architecture-17464e6cbf30?source=rss----3a8144eab

https://cdn-images-1.medium.com/max/1024/1*nWy5K8Hc8Q0CBsrFE8tO1w.jpeg

What is Zero-Trust Architecture and why is it Relevant Today?Breaking Down Zero Trust Architecture:Zero Trust architecture is an option to be considered by organizations who want a more reliable way of preventing leaks of confidential data and lowering the risk of modern #cyber-attacks against their network. Zero Trust was initially developed by the analytics firm, Forrester Research and it is marketed as an alternative to traditional #security architectures.The majority of businesses use traditional security architecture, which functions using the now incorrect theory that anything which is contained within their own network can be considered trustworthy. In the modern day, security threats can arise internally and with even more intelligence than ever before.New security methods need to (...)

#cloud-computing #cybersecurity

#serverless Computing with #swift
▻https://hackernoon.com/serverless-computing-with-swift-4fa27ab36161?source=rss----3a8144eabfe3-

https://cdn-images-1.medium.com/max/974/1*h5JHIRJxUFLMwSCMuSoxpg.png

Part 2: Apache OpenWhisk and IBM CloudYou’d Think I’d Worry About Baking Bread…Recall from Part 1 of this post — Serverless Computing with Swift: Swift and AWS Lambda — that I have decided to form a startup, It’s The Yeast I Can Do, an online bakery specializing in gourmet bread. My CLO (Chief Leavening Officer) is willing to go along with my prioritizing cloud infrastructure over actually baking product. But she’s concerned that we haven’t fully explored our options. In this article, I’ll re-write the code from Part 1 to run under Apache OpenWhisk.OpenWhisk is an open source platform for serverless computing. You can host it on your own servers, with or without containers (such as Docker). You can even run it on AWS. For this article, I will use IBM Cloud, the cloud services infrastructure formerly (...)

#cloud-computing #mobile-app-development #swift-programming

Making Terraform and #serverless framework work together
▻https://hackernoon.com/making-terraform-and-serverless-framework-work-together-b00e6af63ee9?sou

https://cdn-images-1.medium.com/max/1024/0*YaJLkH9ZMlrnCLMO.png

The Serverless framework is the most popular deployment framework for serverless applications. It gives you a convenient abstraction over CloudFormation and some best practices out-of-the-box:Filters out dev dependencies for Node.js function.Update deployment packages to S3, which lets you work around the default 50MB limit on deployment packages.Enforces a consistent naming convention for functions and APIs.But our serverless applications is not only about Lambda functions. We often have to deal with share resources such as VPCs, SQS queues and RDS databases. For example, you might have a centralised Kinesis stream to capture all applications events in the system. In this case, the stream doesn’t belong to any one project and shouldn’t be tied to their deployment cycles.You still need (...)

#cloud #cloud-computing #aws-lambda #aws

Recent database offerings by AWS -Good for users, Dangerous for open source business models
▻https://hackernoon.com/recent-database-offerings-by-aws-good-for-users-dangerous-for-open-sourc

https://cdn-images-1.medium.com/max/760/1*kW9hKMgg5-oPkuriR6yWHg.png

Recent database offerings by AWS — Good for users, Dangerous for open source business modelsAmazon Web Services (AWS) is one of the leading cloud providers who offer a range of cloud services to their customers under their portfolio. Although many of the AWS cloud service offerings are built in-house, some of them are fully managed open source software solutions. In addition, few of there in-house service offerings are also compatible with well-known open source compatibility drivers, interfaces & etc.Open Source and Managed Services in CloudOver the past decade, leading cloud providers including AWS has moved their focus in providing higher-level services. These include not only cloud-specific services but also popular open source software offerings.You might be wondering whether this (...)

#saas #competition #cloud-computing #business-models #open-source

The 10 Most In-Demand IT Skills and Certifications of 2019
▻https://hackernoon.com/the-10-most-in-demand-it-skills-and-certifications-of-2019-241242de685b?

https://cdn-images-1.medium.com/max/1024/1*6-hINfela9qfAmncjJ6Skw.png

Level up your career with the qualifications tech companies are looking for this yearThe job market today is very tough, and the best way to enhance your credibility is getting a few strategic certifications behind your name.Project Management Certifications:Project Management certifications are having an huge demand in the IT market.It is considered to be one of the most important task across industries. It requires special skills for delivering a successful project not as an individual but as an organisation. To Judge your expertise there are wide range of project management certifications are available in the market. They are Certified Scrum Master, Six Sigma Green Belt, Project management Professional, PRINCE2. To acquire these certifications there are some (...)

#big-data #ethical-hacking #devops #cloud-computing #artificial-intelligence

Understanding the scaling behaviour of #dynamodb OnDemand tables
▻https://hackernoon.com/understanding-the-scaling-behaviour-of-dynamodb-ondemand-tables-80d80734

https://cdn-images-1.medium.com/max/1024/0*_aOv3BPcNNx8hI2m.png

Back when #aws announced DynamoDB AutoScaling in 2017, I took it for a spin and found a number of problems with how it works.At re:invent 2018, AWS also announced DynamoDB OnDemand. DynamoDB OnDemand tables have different scaling behaviour, which promises to be far superior to DynamoDB AutoScaling.Talking with a few of my friends at AWS, the consensus is that “AWS customers no longer need to worry about scaling DynamoDB”! So I ran a number of experiments to help me understand how OnDemand tables deal with sudden spikes in traffic.From 0 to 4000, no problem!Let’s assume that a newly created table can handle X amount of read/write throughput before it needs to scale. We can find out the value of X by:create a new tablegenerate a traffic that ramps up to Y writes/s throughput in one minute (using (...)

#cloud-computing #scalability #cloud

What is #aws #lambda or #serverless ?
▻https://hackernoon.com/what-is-aws-lambda-or-serverless-f0a006e9d56c?source=rss----3a8144eabfe3

https://cdn-images-1.medium.com/max/800/1*-P0w5Fgk5Ixj_3IEmjAL7g@2x.png

AWS Lambda logoThere has been a lot of talk about serverless and AWS lambda these days. More and more companies and teams are switching towards the serverless architecture for their applications.In this article, I will be explaining serverless and we will be deploying our first hello world lambda function!What is Serverless?Serverless simply means that you don’t have to manage the servers on which your application runs.You don’t have to take care of patching the system, installing antivirus software or configuring firewalls. Also, you don’t have to worry about scaling your application as the load increases, it is handled automatically!Hence, it allows you to focus more on the functionality of your application.Why is it better?Normally, when you are creating a dynamic web application you (...)

#cloud #cloud-computing

Stretching the Mold: How Alibaba Enhances #database Flexibility
▻https://hackernoon.com/stretching-the-mold-how-alibaba-enhances-database-flexibility-78ec97fcfd

https://cdn-images-1.medium.com/max/1024/1*VVS0u71gPDArGCEpr4Kaxg.jpeg

From hybrid cloud flexibility to containerization, supporting the 11.11 Global Shopping Festival has taken relentless innovation from Alibaba’s database team.For Alibaba Group, preparing to support peak data traffic during the annual 11.11 Global Shopping Festival is a yearlong challenge, at the center of which demand for database flexibility has been a perennial feature.As a widely used data storage system, Alibaba’s database consumes IO and CPU resources to perform operations involved with its SQL requests, including physical read, logical read, and sorting and filtering. With different services’ SQL requests consuming different resources according to their execution plans, the demand for resource specification varies by service.To improve overall utilization, Alibaba must abstract (...)

#containers #cloud-computing #docker #database-flexibility

The $300 Lettuce : Building a Smart Garden
▻https://hackernoon.com/the-300-lettuce-building-a-smart-garden-c048c0e1a51?source=rss----3a8144

https://cdn-images-1.medium.com/max/776/1*YihMlrGPajr22Wx6qmO_hw.png

A few months ago, a very generous friend of mine gave me a fantastic gift, my very own Raspberry Pi 3B+. For those who are unfamiliar, a Raspberry Pi is a small and affordable computer which can also be connected to sensors and other electronics.The first challenge was deciding what to do. I’m always looking to extend my skills as a software developer, and I’d been toying with the idea of growing some vegetables. Receiving the Raspberry Pi was exactly the impetus I needed to start a project: a smart hydroponic garden system, with automated alerts, data analysis and maintenance.Spoiler Alert: If you’re only looking to grow food, this project isn’t particularly efficient. Applying computing to a small home garden is time consuming, sometimes costly, and doesn’t necessarily improve yield. (...)

#raspberry-pi #aws #cloud-computing #hydroponics #iot

What is Amazon Lightsail?
▻https://hackernoon.com/what-is-amazon-lightsail-beaef47dd64e?source=rss----3a8144eabfe3---4

https://cdn-images-1.medium.com/max/1024/1*QiHBlwnNjmRVnF7EiVsviQ.jpeg

Amazon Lightsail LogoIf you are new to #aws and looking to deploy some servers for your applications then AWS Lightsail may be the best starting point for you.Unlike Amazon EC2, you are given a nice interface where you can select preconfigured plans that may cover most of your use cases.Just with 3–4 clicks you can launch a WordPress website running on Linux server.You also don’t have to worry about determining the cost since the cost is fixed monthly.Along with servers, you can also create databases, load balancers, and storage on Lightsail.Now let’s get practical and quickly launch a Node server with Lightsail!Launching an InstanceFirst, log into your AWS console and in the All Services tab under Compute you will find Lightsail. Click on it, it will open the Lightsail dashboard in a new (...)

#amazon-lightsail #cloud-computing #computer-science #software-development

Chaos test your Lambda functions with Thundra
▻https://hackernoon.com/chaos-test-your-lambda-functions-with-thundra-48743fe98a91?source=rss---

Failures are inevitable. Just as we need to test our application to find bugs in our business logic before they affect our users. We need to test our application against infrastructure failures. And we need to do it before they happen in production. and cause irreparable damages.The discipline of Chaos Engineering shows us how to use controlled experiments to uncover these weaknesses.In this post, we will see how we can leverage Thundra’s span listeners to inject failures into our #serverless application. We will use these failures to expose weaknesses such as:Missing error handler for DynamoDB operations.Missing fallback when the primary data source is unavailable.Missing timeout on outbound HTTP requests.But first, here’s a quick primer on Thundra.Hello, Thundra!Consider an API with two (...)

#lambda-function #aws #cloud-computing #aws-lambda

How to Reduce Costs And Increase Efficiency Of Cloud Service Development
▻https://hackernoon.com/how-to-reduce-costs-and-increase-efficiency-of-cloud-service-development

Image credit: UnsplashEvery cloud service consists of several components. Therefore, the functionality accessed by the end user is just the tip of the iceberg. There are also a lot of infrastructure-related features that are invisible to the users but crucial to getting the solution up and running.Under the hood of the cloud serviceRegistration of a new tenant of a cloud service is a perfect example of such a feature. If we are talking about a CRM system or accounting software, the user cannot just log in to the system. Usually, before this, they land on a website, sign up for the service, confirm registration via email, log in to the site, choose a pricing plan, pay for it (or chose a free demo plan), and finally get to the application.Once the plan is chosen, we need to create a tenant (...)

#guides-and-tutorials #software-development #cloud-service-development #cloud-computing #cloud-services

What Does the Future of Cloud Computing Mean for Media?
▻https://hackernoon.com/what-does-the-future-of-cloud-computing-mean-for-media-c4894d0bbaa?sourc

https://cdn-images-1.medium.com/max/1024/1*YfIX34PPEEcBOY6i9gUVIQ.png

Would it surprise you to hear that 73% of companies run at least one application in the cloud? Maybe the only surprise is that the percentage isn’t higher.This is especially true for media and entertainment companies in segments such as publishing, broadcasting, music, and sports. The amount of content we now produce, stream, watch online or download to our devices is mind-blowing. Each day, 2.5 quintillion bytes of data are generated in the digital sphere.For example, every minute in 2017, Spotify added 13 new songs, Wikipedia users published 600 new page edits, Instagram users posted 46,740 pictures, Netflix viewers streamed 69,444 hours of video, and YouTube users watched 4,146,600 videos.In the increasingly digitized era of growing connectivity, cloud data storage and cloud computing (...)

#media-and-entertainment #cloud-services #cloud-storage #cloud-computing

Lambda optimization tip — enable HTTP keep-alive
▻https://hackernoon.com/lambda-optimization-tip-enable-http-keep-alive-6dc503f6f114?source=rss--

https://cdn-images-1.medium.com/max/1024/0*bmImHeGBqn0JWrQz.png

Lambda optimization tip — enable HTTP keep-aliveI recently watched an excellent talk by Matt Levin on optimization tips for Lambda and saw a slide on making DynamoDB use HTTP keep-alive. It reminded me of a conversation I had with Sebastian Cohnen, so I set out to test the effect this simple optimization has.What is it all about?As it turns out, Node.js’s default HTTP agent doesn’t use keep-alive and therefore every request would incur the cost of setting up a new TCP connection. This is clearly inefficient, as you need to perform a three-way handshake to establish a TCP connection. For operations that are short-lived (such as DynamoDB operations, which typically complete within a single digit ms) the latency overhead of establishing the TCP connection might be greater than the operation (...)

#cloud #serverless #aws #cloud-computing #aws-lambda

Subscribe SQS to a SNS topic in another #aws account with #cloudformation, and gotchas!
▻https://hackernoon.com/subscribe-sqs-to-a-sns-topic-in-another-aws-account-with-cloudformation-

https://cdn-images-1.medium.com/max/900/0*_H3-LaaQUzj4JB_g.png

A use case arrived recently, where we need to subscribe a SQS queue to a SNS topic running in another AWS account. On the surface this seems like something many people would need to do, and indeed I was able to find an official tutorial pretty quickly. But the tutorial is all “click this in the SQS console, and do that in the SNS console”. We are strong believers in Infrastructure as Code and having someone do the subscription steps manually is not going to work, especially given that we’ll be repeating this process in many places.So here’s what I learnt as I translate the tutorial steps into CloudFormation, and some gotchas I found.First, in the SNS account, you need to add a SNS TopicPolicy to give the SQS account permission to call sns:Subscribe on the relevant topic(s).Then, in the SQS (...)

#sqs-to-a-sns-topic #cloud-computing #cloud

Securing The Cloud
▻https://hackernoon.com/securing-the-cloud-b38e1fbca0f8?source=rss----3a8144eabfe3---4

https://cdn-images-1.medium.com/max/768/1*l1s6_YMm3kZnVwp3fPFWRQ.png

Over the years the cloud has become a cost-effective way to hold data. The cloud is a great way to store files while allowing your employees easy access quicker than passing them a USB drive or sending an email. Employees can be located anywhere in the world and still have access to necessary files.However, with the cloud come risks when it comes to securing your data away from unwanted eyes. Cloud security is the utmost importance to ensure that your information isn’t getting into the wrong hands. This is especially true if the data stored needs to follow HIPAA laws. Here are some tips for keeping your data safe and increasing your cloud security.Don’t Store Sensitive InformationThe minimal thing you can do to keep your cloud information secure is as simple as not storing sensitive (...)

#nist-framework #secure-the-cloud #cloud-computing #securing-the-cloud #cloud-security

Open Season for Research: Alibaba Releases Cluster Data from 4000 Servers
▻https://hackernoon.com/open-season-for-research-alibaba-releases-cluster-data-from-4000-servers

https://cdn-images-1.medium.com/max/1024/1*Qxo2Yg8ezFfbgbhsE1qWVg.jpeg

To help researchers and developers in academia and beyond, Alibaba has now published its Cluster Data V2018 for all to exploreIn IT articles about internet applications, you will often find words such as “large scale” and “mass requests”. These applications all run in large-scale data centers, and readers generally have many questions about those environments. For example, what is the operating status of each machine in the data center? What kinds of applications are running? What are the characteristics of these applications? With the exception of a few senior experts, it is rare for students and corporate researchers to understand these details.Today, Alibaba shares a real #dataset for computer clusters: Alibaba Cluster Data V2018 (attached at the end of article). This dataset provides a (...)

#cloud-computing #data-center #big-data #software-development