#python

An open-(source, science) tool to extract tables from PDFs into Excels
▻https://hackernoon.com/an-open-source-science-tool-to-extract-tables-from-pdfs-into-excels-3ed3

https://cdn-images-1.medium.com/max/1024/0*8YsOjqB-FQPkCAlY.png

I originally wrote this post for my website.Photo by Patrick Tomasso on UnsplashBorrowing the first three paragraphs from my previous blog post since they perfectly explain why extracting tables from PDFs is hard.The PDF (Portable Document Format) was born out of The Camelot Project to create “a universal way to communicate documents across a wide variety of machine configurations, operating systems and communication networks”. Basically, the goal was to make documents viewable on any display and printable on any modern printer. PDF was built on top of PostScript (a page description language), which had already solved this “view and print anywhere” problem. PDF encapsulates the components required to create a “view and print anywhere” document. These include characters, fonts, graphics and (...)

Adaptive trend following #trading strategy based on Renko
▻https://hackernoon.com/adaptive-trend-following-trading-strategy-based-on-renko-9248bf83554?sou

https://cdn-images-1.medium.com/max/1024/1*OGRZUK5xFunyqJULbmyjqA.png

Today I’m going to show how to create an algorithmic trading strategy on #python. This strategy uses my original research from one previous article. This current article consists of these parts:ConceptAlgorithm descriptionTrading strategy developmentBacktesting and analyzing the resultFurther problems discussionConclusionsConceptFinancial time-series have a high level of noise in data. Would be good to have an ability to reduce a noise. In this article it is proposed to use Renko brick size optimization. The key idea of the approach is to quantify the quality of a Renko chart and try to get an optimal brick size for using in a trading. If you are not familiar with the Renko charts will be better follow the link of the article.The optimization of quality over time is called an “adaptivity” (...)

#stock-market #cryptocurrency #software-development

Efficient development with #docker and docker-compose
▻https://hackernoon.com/efficient-development-with-docker-and-docker-compose-e354b4d24831?source

https://cdn-images-1.medium.com/max/1024/1*u7fb5vghcVeF5S2DYxrhiA.jpeg

We are going to set up a development environment for a project consisting of various services. All of these services will be containerized with Docker, and they will all run simultaneously during development using docker-compose.Our environment will feature instant code reloading, test-driven development, database connectivity, dependency management, and more™. It will be possible to easily deploy it to production with docker-compose or with Rancher. And as a bonus, we’ll set up continuous integration on Gitlab.The article is about efficiency, so I’ll get straight to the point.The goalsWe want to:Type code and observe changes as soon as possible in our Docker containers, preferably without manual actions;Have a local environment that is representative of the actual deployment (...)

#pytest #gitlab #python #java

A repository with the example of printing “Hello World” in a lot of programming languages
▻https://hackernoon.com/a-repository-with-the-example-of-printing-hello-world-in-a-lot-of-progra

https://cdn-images-1.medium.com/max/1024/0*4ty0Adbdg4dsVBo3.png

Img source: yen.ioHello world!One of the first things that you are instructed to do when you start to learn to programme, or when you are simply learning a new programming language, is doing something really simple. One of the first steps that you do is print a simple text. As you are reading this, you probably may know which text is the most common and the one that I am referring to.The text that I am talking about is Hello World.This text is supposed to be for you as a way of greeting the world as you have just embarked on an important journey. A journey of learning as much as possible.A #github repository has already been prepared with taking examples from as many programming languages and putting them in a commonplace so that other people can see how this same procedure is done in (...)

#python #hello-world #repositories #java

#Formation #Initiation #Python à Nantes du 10 au 12 décembre
▻https://makina-corpus.com/blog/formation/2018/formation-initiation-python-nantes

Vous êtes développeur et maîtrisez déjà un langage de programmation ? Python vous tente et vous avez besoin d’un coup de pouce pour bien commencer ?

#Paris #News_Item

Tired of bookmarking pages? Scrap it instead …
▻https://hackernoon.com/tired-of-bookmarking-pages-scrap-it-instead-b398622f77d?source=rss----3a

https://cdn-images-1.medium.com/max/1024/1*qr-F-zyqFa1UATZcl0SrsA.png

Tired of bookmarking webpages? Scrap it instead …I am guilty of bookmarking a ton of tutorials but never opening them again. I am one of those who prefer a pdf version or a book over a mountain of web pages as bookmarks or tabs stored in OneTab.I was refreshing my Operating Systems concepts recently from my favorite site ▻https://www.geeksforgeeks.org/operating-systems/.Though I loved learning OS, what I didn’t like was having to open weblinks one after other according to the topic. I wanted to have all the content of the pages as a single file for my perusal. Following that was my attempt at automating the process of ‘extracting text from websites’ aka web scraping. The use cases for web scraping are limitless. Like checking for flight fares, monitoring the stock prices, product prices (...)

#python #programming #web-development #software-development #web-scraping

How to make wise choices in selecting #libraries
▻https://hackernoon.com/how-to-make-wise-choices-in-selecting-libraries-4f292a7247cd?source=rss-

https://cdn-images-1.medium.com/max/1024/1*cxVFO11rQzA_QjbQ2yNt8Q.png

Choosing a library is like choosing a wife, you have to stick with it and often changing one involves complicated process that is very expensive. So the best way to do this by analysis.Now a days most analysis of libraries involve the features of the library. So we won’t get into that. We will go through this problem by example choosing a model server library for machine learning. A model server library allows you to easily make a server for your machine learning model.Select the library which gives you the most featuresThis is a fairly obvious choice picking the one that gives you the most features. You look at your requirements and you see the library which gives you the minimal features you need.So i have shortlisted 4 libraries:Tensorflow servingClipper.aiModel Aerver for Apache (...)

#github #bigquery #python

#python fundamentals (3) — handling variables
▻https://hackernoon.com/python-fundamentals-3-handling-variables-552032810de3?source=rss----3a81

https://cdn-images-1.medium.com/max/863/1*hHStoxKCgww4pg6uU8r5JQ.png

Python fundamentals (3) — handling variablesThere are many metaphors for variables in Python. One could view a variable as something like a bucket, into which you can place data. So the bucket called age could have the value of 12 — but, crucially, because it is a variable, it’s value can change to 13 or any other number at any point.In truth, most quantities that we use in a #programming language can have values that change, and these quantities are variables (the opposite of which is a constant, but more on that another time). A variable in Python has an identifier, and the Pythonesque way of thinking of this is (note the lower case):identifier = valueSo for example: age = 13…has age as the identifier, in this case a variable, and 13 as the value. Another example:name = input (“What is your (...)

#python3 #coding #edtech

#python fundamentals (2) — saving Python programs as files
▻https://hackernoon.com/python-fundamentals-2-saving-python-programs-as-files-a7932d4dfe85?sourc

https://cdn-images-1.medium.com/max/1024/1*hquBR03PXaM5a9-JzO1ZNA.png

Python fundamentals (2) — saving Python programs as filesIDLE has a mode in which the user can write multiple lines of code, save them in a file, and execute them whenever he/she chooses. This is, well, pretty handy if you are #programming:Here you can see that I am in a New File (right window)- and that the executed code is running in the left window. Quirk of Python — writing 17 + 9 and 67–14 is absolutely fine; it returns no error message. However it does not display in the shell (also known as the console*) — you need to actually issue Python with a print command in order to see the results of the calculation. So Python will obediently hold onto the results of calculations — possibly storing them in variables or as arrays or something — but will steadfastly keep hold of them until you issue a (...)

#edtech #coding #python3

10 choses qui me font aimer #Python
▻https://makina-corpus.com/blog/metier/10-choses-qui-me-font-aimer-python

La 7e va vous surprendre ! Haha, non en fait il fallait que je m’arrête à 10, sinon vous ne liriez pas cet article de par sa longueur...

#News_Item

11 Ways to Make #python a Dangerous Language For Data Science
▻https://hackernoon.com/11-ways-to-make-python-a-dangerous-language-for-data-science-cc51e0dea1a

https://cdn-images-1.medium.com/max/1024/1*kJ7UuQJh0q3qcv8wsTsk8Q.jpeg

People are still crazy about Python after twenty-five years,which I find hard to believe.WordCloudPython has numerous applications — web development, desktop GUIs, software development, business applications and scientific/numeric computing. In this series we will be focusing on how to use numeric computing in Python for data science and ML.This is not a comprehensive Python tutorial but instead is intended to highlight the parts of the language that will be most important to us(some of which are often not the focus of Python tutorials).In this tutorial, we will be looking at the following basic features of Python :Whitespace FormattingPython function3. Data types and sequences4. Date and time5. Lambda6. Map7. Filter8. Reduce9. Zip10. For loop11.Control Flow12. List comprehension1.Whitespace (...)

#dangerous-data-science #python-data-science #data-science #machine-learning

How to backup personal #github repositories
▻https://hackernoon.com/how-to-backup-personal-github-repositories-f42235185806?source=rss----3a

I will show how to do a backup of your GitHub repositories with #python-github-backupWhy to bother with a backup of GitHubI can already see that there will be comments regarding why to do the backup of GitHub.“It is a waste of time.”“GitHub internally already have backups.” (I hope so)“They will not lose your code” (But maybe I will)“They will not go overnight out of business.”Response to all those comments is: You will not be worst off if you have your own backup.If forever reason (GitHub go under, all repositories deleted by accident, alien attack) GitHub is not available anymore, I have my own backup of code that I have written.Paid solutionIf you are looking for a paid solution, BackHub looks like a good solution. I have no experience with BackHub, nor am I in any way associated with it.Free (...)

#github-repositories #backup-github-repos #backup-github

A simple introduction to #python’s #asyncio
▻https://hackernoon.com/a-simple-introduction-to-pythons-asyncio-595d9c9ecf8c?source=rss----3a81

https://cdn-images-1.medium.com/max/1024/1*R4HzblupF0U1sgfCqH9SUg.png

This is a no-buzzword first principles introduction to the asyncio library in Python.If you’ve come here, it is likely that you have heard of words such as asynchronous, #concurrency and parallelism. Before we start off with asyncio, lets quickly get some basic things about these words right (via examples), so that we have a solid foundation to build this upon.Concurrency is like having two threads running on a single core CPU. Instructions from each thread could be interleaved, but at any given time, only one of the two threads is actively making progress.Parallelism is like having two threads running simultaneously on different cores of a multi-core CPU.It is important to note that parallelism implies concurrency but not the other way round.Asynchronous is a higher level #programming (...)

#python3

#python Tricks 101
▻https://hackernoon.com/python-tricks-101-2836251922e0?source=rss----3a8144eabfe3---4

https://cdn-images-1.medium.com/max/1024/1*Lmq0zAoUpexi9aAg1qR_rA.png

Python Tricks 101?Python tips which are relatively uncommon and are very useful.Talk is cheap show me the codeSwapping valuesCreate a single string from all the elements in listFind The Most Frequent Value In A List.Checking if two words are anagramsReverse a StringReverse a listTranspose 2d arrayChained ComparisonChained function callCopying ListDictionary GetSort Dictionary by ValueFor ElseConvert list to comma separatedMerge dict’sMin and Max index in ListRemove duplicates from a listIf you think i should i add anymore or have suggestions please do comment. i’ll keep on updating this blog.Most of the tricks and tips are taken from PyTricks and a few blogs .Thank you for reading ?. If you like the Article give it a clap ?.If you wish to have a chat, DM me at ▻https://twitter.com/gauthamzzz.I am a (...)

#python-tricks #python-tricks-101 #programming #show-me-the-code

#Formation #Initiation #Python à #Paris du 10 au 12 septembre
▻https://makina-corpus.com/blog/formation/2018/formation-initiation-python-paris

Vous êtes développeur et maîtrisez déjà un langage de programmation ? Python vous tente et vous avez besoin d’un coup de pouce pour bien commencer ?

#News_Item

ML Experiment & Epic Failure
▻https://hackernoon.com/ml-experiment-epic-failure-fe0940f0bb51?source=rss----3a8144eabfe3---4

https://cdn-images-1.medium.com/max/300/0*931VXTMBG18pmw_G.png

I always love to write about my failed experiments so that people who experiment a lot will know that they are not alone. This blog is one of those ML experiments that show how not to do certain things.From the moment ML gained its hype, many of us jumped in and started learning it. Traversing from Andrew Ng to Siraj Raval, linear algebra to deep learning and neural nets, I have seen it all, but it didn’t get me anywhere. I haven’t hit my eureka moment even after going through series of tutorials and folders full of ML projects.For you to hit the magic moment of where it all makes sense to you, you have to see it working the magic in front of you, which never really happened neither to the ML models I created nor to me.Stepping stones to MLGet the datasetPreprocess — Cleaning, fillNaN, (...)

#sklearn #deep-learning #python #ml-experiment #machine-learning

Introduction of #tensorflow with Python
▻https://hackernoon.com/introduction-of-tensorflow-with-python-f4a9624f2ab2?source=rss----3a8144

https://cdn-images-1.medium.com/max/1024/1*JGDhEOqWVmzqofJ7xSS0KA.jpeg

Photo on UnsplashMachine learning is the most popular part of our technology world. TensorFlow is also a popular open source and it is a framework of deep learning. The Deep Learning is part of Machine Learning.Photo on UnsplashDeep learning is also a large part of machine learning methods based on learning data presentations — “as opposed to task-specific algorithms.”Photo on @GoogleWhat is TensorFlow?It’s a framework to perform computation very efficiently, and it can tap into the GPU (Graphics Processor Unit) in order too speed it up even further. This will make a huge effect as we shall see shortly. TensorFlow can be controlled by a simple Python API, which we will be using in this Article.Graphs and TensorsWhen a native computation is done in many programming languages, it is usually (...)

#matplotlib #python3 #machine-learning #deep-learning

Build a chat app in the terminal using #python
▻https://hackernoon.com/build-a-chat-app-in-the-terminal-using-python-9f306e57db8f?source=rss---

https://cdn-images-1.medium.com/max/600/1*nX0_idjq0t0CiRs7Vfe-Rg.gif

Pusher, our weekly sponsor, makes communication and collaboration APIs that power apps all over the world, supported by easy to integrate SDKs for web, mobile, as well as most popular backend stacks. Get started.Realtime chat is virtually any online communication that provides a realtime or live transmission of text messages from sender to receiver. This tutorial will show you how to build a realtime terminal chat using Python and Pusher Channels.It’s lightweight to use the terminal for our chat, as there is no opening of the browser, loading of JS libraries or any frontend code. Also, it allows us to quickly test our ideas without worrying about what the user interface would look like.Python in this tutorial refers to Python 3.xPrerequisitesA basic understanding of Python is needed to (...)

#build-a-chat-app #weekly-sponsor #chat-apps #python-chat-app

A Hands-On Introduction to Neural Networks
▻https://hackernoon.com/a-hands-on-introduction-to-neural-networks-6a03afb468b1?source=rss----3a

https://cdn-images-1.medium.com/max/1024/1*PSEHaQtQ3D6Bq959ytYcDQ.jpeg

Implementing a Single Neuron in #python from ScratchIn the last decade, Artificial Intelligence (AI) has stepped firmly into the public spotlight, in large part owing to advances in Machine Learning (ML) and Artificial Neural Networks (ANNs).But with promising new technologies comes a whole lot of buzz, and there is now an overwhelming amount of noise in the field. That’s why I thought it would be useful to get back to basics and actually implement a single neuron from scratch using Python.The Artificial NeuronBefore we dive in, I just wanted to quickly talk about what a neuron is in the first place. Early proponents of artificial intelligence noticed that the biological neuron was capable of conceptualizing and learning from large volumes of data, and postulated that modelling this neuron (...)

#neural-networks #deep-learning #programming #machine-learning

Don’t Trust a Pickle
▻https://hackernoon.com/dont-trust-a-pickle-a77cb4c9e0e?source=rss----3a8144eabfe3---4

https://cdn-images-1.medium.com/max/1024/1*tl38poVdSX74pGSa217bIA.jpeg

Don’t Trust a PickleIf you are using #python, especially for machine learning, you should be somewhat familiar with the standard library module named pickle. It is used for Python object serialization and comes very handy in wide range of applications. Some objects that you might want to serialize: a trained scikit-learn model, a Pandas DataFrame that you got after a lengthy join of several tables; basically any Python object that consists of heterogeneous data that you might want to quickly load in a new environment in the future (for homogeneous data, like neural network weights or training data tensor, it’s better to use a more suitable format like HDF5).In this article I would like to tell you why you should be very cautious when unpickling an object that you obtained from an untrusted (...)

#python-pickl #dont-trust-a-pickle #pickles #programming

Scaling Effectively: when #kubernetes met Celery
▻https://hackernoon.com/https-medium-com-talperetz24-scaling-effectively-when-kubernetes-met-cel

https://cdn-images-1.medium.com/max/1024/1*r6VX5egi8sI1XKSPY9a5-g.png

This is a story about software architecture, about a personal itch, and about scalability. And like any good tech story, it begins with a shaky architecture.At Panorays, we help large enterprises to measure the security posture of their suppliers. But I’m not going to get into the whole 3rd party security management extravaganza with you. we came to talk about our architecture and process.In the beginning, there was bash. and scripts to manage VMs. a lot of scripts.There was a VM instance for each company we assessed.Every VM executed sequential batch jobs that imitate the whole reconnaissance phase of the hacker’s lifecycle.Company level parallelism is achieved by firing up more VMs.We built an internal orchestration system via Cron & Bash (imagine how fun was that…).Problems:The (...)

#startup-lessons #kubernetes-and-celery #docker #python

How to develop a Telegram chatbot on #python
▻https://hackernoon.com/how-to-develop-a-telegram-chatbot-on-python-acda22be3059?source=rss----3

https://cdn-images-1.medium.com/max/800/1*bKS8ju5xzryMbOiVejYuDg.png

Technological progress and automation are starting to influence numerous spheres of human economy and everyday life. The rapid development of artificial intelligence imposes training computers to do the human work and implement their usage in business. One of the main applications of artificial intelligence in business is the chatbot.NLP together with #chatbots have great potential in the area of customer service, and can easily accept the customer’s order and give them a consultation regarding the company’s services.It is possible to automate the work of support centers with the help of bots on the official website of the company or in popular messengers like Telegram, Slack or Facebook Messenger.In this article, we shall give you a brief tutorial about chatbot development, and share our (...)

#chatbot-development #python-programming #machine-learning

#pandas cheatsheet for #sql people (part 1)
▻https://hackernoon.com/pandas-cheatsheet-for-sql-people-part-1-2976894acd0?source=rss----3a8144

Pandas library is the de-facto standard tool for data scientists, nowadays. It is used widely by many data scientists around the globe. After being familiar with it I always use it for processing table-structured data whatever project I am working on. It works fast and reliable, supports CSV, Excel, JSON and so on.However, as a person experienced in SQL, I had some difficulties and confusion with manipulating the tables (a.k.a. #dataframes) in the beginning. Eventually, I learned more APIs and ways of doing the things properly. I believe many people who do his/her first steps on Pandas may have the same experience. Some time ago I prepared the cheatsheet using SQL queries and their analogy in Pandas. And I am happy to share it with all of you.In this post, I am sharing the queries using (...)

#python #data-science

A Common Misunderstanding About #python Generators
▻https://hackernoon.com/a-common-misunderstanding-about-python-generators-dbc622914d33?source=rs

I received the following email a few days ago:Jeff,It seems that you know about iterators. Maybe you can explain some weird behavior. If you run the code below you will find that the function is treated differently just because it has a ‘yield’ in it somewhere, even if it’s completely unreachable.def func(): print("> Why doesn’t this line print?") exit() # Within this function, nothing should matter after this point. The program should exit yield "> The exit line above will exit ONLY if you comment out this line."x = func()print(x)When I run the code, I get the following output from the print() call: <generator object func at 0x10e968a50>.So what’s going on here? Why doesn’t that line in func() print? Even if yield is completely unreachable, (...)

#tutorial #python-generators #programming #how-to

Visualizing Linear Regression with #pytorch
▻https://hackernoon.com/visualizing-linear-regression-with-pytorch-9261f49edb09?source=rss----3a

https://cdn-images-1.medium.com/max/864/1*NO-YvpHHadk5lLxtg4Gfrw.gif

Linear regression is a common machine learning technique that predicts a real-valued output using a weighted linear combination of one or more input values.For instance, the sale price of a house can often be estimated using a linear combination of features such as area, number of bedrooms, number of floors, date of construction etc. Mathematically, it can be expressed using the following equation:house_price = w1 area + w2 n_bedrooms + w3 n_floors + ... + w_n age_in_years + bThe “learning” part of linear regression is to figure out a set of weights w1, w2, w3, ... w_n, b that leads to good predictions. This is done by looking at lots of examples one by one (or in batches) and adjusting the weights slightly each time to make better predictions, using an optimization technique (...)

#visualization #linear-regression #machine-learning #python