So, my idea was to obtain all the elements, tagged with “li” from here – vitoshacademy.com/all, using BeautifulSoup4 and Python. Initially, I thought about running a simple soup.findAll("li") , but it also added the text from the menus, which were…
Category: Python
Python code examples
Scraping a web site up to the N-th level with Python
Scraping data from a web site has always been a pleasure with Python. If you come from the VBA world, the BeautifulSoup4 is indeed beautiful! I have been scraping a web site with python before, but now as I have…
Prime numbers matrix and coloring with Python and xlsxwriter
About 4 years ago, I wrote an article about prime numbers in Excel and their visualisation with VBA – vba-function-in-excel-to-check-prime-numbers. Pretty much, it displays the prime numbers in a 10 x N matrix, starting from 1 and finishing at N:…
Tagged with: align, bg_color, center, Excel, first column and first row of Excel is with index 0, font_color, format, matrix, prime numbers, Python, xlsxwriter
Scraping web site and writing its links to a database with Python
Scraping a web site with BeautifulSoup 4 and Python is a walk in the park, if you have scraped a web site with VBA before. Still, there are a few tricks, that should be taken into account. Trick #1 –…
Adding data to SqlLite3 database with Python
After reading the database to python from the previous post here, it is time to do the ADD data to the database. Our database is created like this:
|
1 2 3 4 5 |
CREATE TABLE "drummers" ( "Id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, "Name" TEXT, "Band" TEXT ) |
The database has autoincremented primary key, thus we do not need…
Connecting SqlLite3 database to Python and Reading it
Connecting SqlLite3 database to a Python file is actually 1 line of code: conn = sqlite3.connect(r"C:\Users\database.db") I have used a database from sqllitetutorial.net, which was rather neat. Then DB Browser for SQLLite a LEFT JOIN was implemented, in order to unite the…
Adding data to SqlLite3 database, but only one value with Python
Life is hard, when you are learning Python and you see things that are making no sense the way they run. After writing about adding data to SqlLite3 with Python, today I have tried to do the same, but for…
Find all HTML tags in a web page and print them from a sorted dictionary
Finding all HTML tags from a web page and recording these to a dictionary is magically easy task with Python (if you compare it with #VBA), when Beautiful Soup 4 is used. soup.find_all(True) loops through the tags and a simple…
Write Formulas in Excel through Python XlsWriter
Writing formulas with Excel with Python is a technique, that needs some practice, actually. Although XlsWriter has some good documentaion, not a lot is covered on the point, that concerns creating a file with a formula in it. First and…
Writing to a text file in Python from a variable, ignoring encoding problems
Writing to a file in Python is usually a simple 2-liner, which is easy to be written:
|
1 2 |
with io.open("soup.txt", "w") as file: file.write(soup_written) |
However, problems with the encoding may cause some research needed. Concerning the encoding this is a great article from the creator of…
Show all links from a website with Python and BeautifulSoup – Video
Showing all links from a website with Python and BeautifulSoup is actually 6 lines of code – 2 to define the used libraries and 4 to do the job:
|
1 2 3 4 5 6 7 8 |
import urllib.request from bs4 import BeautifulSoup resp = urllib.request.urlopen("https://www.vitoshacademy.com") soup = BeautifulSoup(resp, from_encoding=resp.info().get_param('charset')) for link in soup.find_all('a', href=True): print(link['href']) |
In general, BeautifulSoup comes out of the box with beautiful…
Tagged with: BeautifulSoup4, display links, extract links to file, print links, Python, show all links
Python – The birthday paradox algorithm
The birthday paradox is an interesting problem, mainly because of its somehow “unexpected” results. The problem usually is stated as the following: What is the minimum number of people in a room, in order to have a probability higher than…
Sets in Python – Differece, Symetric Difference, Union, Intersection
After writing the article scraping a web site up to the N-th level with Python, I have decided to summarize what could be achieved with Set() in this language. Pretty much, sets can be described as lists with a few…
Numpy and Pandas – Small introduction with Jupyter Notebook
Last week I was participating at the Python and ML Summit in Berlin. One of the most interesting lectures, on which I participated, was a workshop by, named “Reading all yourself was yesterday – How to turn large amounts of…
Python and MachineLearning Summit in Berlin – Summary
This Monday and Tuesday (14. – 15. October 2019) I was in Berlin, participating at the #Python and #ML Summit. Actually, #Berlin is a nice city, one of my favourites in Germany, thus if you are not from there, only…
Python – Stand Alone Application with GUI
Python has a built-in GUI package, which allows easy building of forms like this: The package used in this example is tkinter. The form above takes the value, entered in the entry field, squares it and displays it back in…
Analyzing trends with map and reduce in Python
Editorial (from Vitosh): Lately, I have reviewed the Mastering Large Datasets book by J.T. Wolohan. The book is python based and it explains how to scale a data project, using the map and reduce to scale data projects. Long story…
Python – Customizing PyLab using Style
Learn how to customize PyLab using style in this article by Benjamin Walter Keller, a PhD candidate at McMaster University with BSc in physics with a minor in computer science from the University of Calgary in 2011. This article will delve…
Python – Plotting with Object-Oriented API
Learn how to plot a correlation matrix using pyplot and object-oriented APIs in this article by Srinivasa Rao Poladi, the co-founder of krtrimaIQ a consulting firm that provides cognitive solutions to create tomorrow’s Intelligent Enterprises powered by automation, big data,…
Fundamental Concepts of Concurrent Programming with Python
Learn some of the fundamental concepts of concurrent programming in this article by Quan Nguyen, a Python enthusiast and data scientist. It is estimated that the amount of data that needs to be processed by computer programs doubles every two years. The International Data Corporation (IDC), for…
Python – Balancing CartPole with Machine Learning
This article will show you how to solve the CartPole balancing problem. The CartPole is an inverted pendulum, where the pole is balanced against gravity. Traditionally, this problem is solved by control theory, using analytical equations. However, in this article, you’lllearn to solve the…
NumPy Fast Operation and Computations
LearnNumPy fast operation and computations in this article by Alberto Boschetti, a data scientist with expertise in signal processing and statistics and Luca Massaron, a data scientist and marketing research director specialized in multivariate statistical analysis, machine learning, and customer…
Build, Test, and Deploy with Zappa
Learn how to build, test, and deploy a Python web service using Zappa in this article by Abdulwahid Abdulhaque Barguzar, a software architect and an active contributor to the open source community. He has developed a deep understanding of architecting…
Using NumPy for Simulations
Learn how to use NumPy for simulations in this tutorial by Alvaro Fuentes, a data scientist with an M.S. in quantitative economics and applied mathematics with more than 10 years of experience in analytical roles. NumPy, also known as Python’s vectorization solution,…
Hashing Passwords in Python
Learn how passwords can be stored without a risk of leaking them in this tutorial by Alessandro Molina, a Python developer since 2001 and currently the core developer of the TurboGears2 web framework and maintainer of Beaker Caching/Session framework.While cryptography is…
Python – Write Values in all Worksheets of existing Excel File
Working with VBA is usually not considered high level of programming. Still, when it comes to Excel, this is probably the best solution. However, Python has some libraries which can help a bit, if you are willing to “mimic” the…
Run Python Functions From Excel With VBA with Shell through CommandPrompt
Selecting Python for a “weapon” of choice has many positive points, which I am not going to describe now. Still, if you want to run python code with the help of VBA and the Shell through the CommandPrompt of Windows,…
How to Search In Excel Worksheet with Python
Working with Python and Excel together is really fun for a VBA Developer! It somehow shows you what all these years of coding with VBA have taught you! Today, I was thinking how to implement the .Find() method of searching in…
Python – Simple writing in Excel with XlsWriter
The XlsxWriter library of Python is a nice gem for a VBA developer – you would be really interested to see the problems a Python developer should be facing, just to achieve a simple part Excel automatization. Still, these are…
DevOps Conference in coming – time to install Python on my machine
In less than 4 days I would be travelling to Berlin to see what is happening on the DevOpsCon there. I will be present for a whole day workshop for machine learning with Python, thus I have decided that it…
Top