Cartopy is a Python package designed for geospatial data processing in order to produce maps and other geospatial data analyses. Making a map with Python and Cartopy is actually easier than you might think, but still it has its own…
Category: Python
Python code examples
Python – Bayes’ theorem – Drug testing example. 99% is not enough!
The Bayes’ theorem is interesting, as far as it is both quite challenging to understand and shocking once you really realize the simplicity of the numbers behind it. It describes the probability of an event, based on prior knowledge of conditions that…
Python – Linear Regression
In 2013 and 2014 (wow, already 7 years ago!) I wrote two articles about linear regression with Excel. Now, I am getting more and more interested in Python, thus I guess it would be interesting to remake the article into…
Tagged with: linear regression, plotting, plotting both predicted and real values, prediction, Python
Python – Building New Columns in a Dataframe, Based on Other Columns
Working with python, pandas and dataframes is quite challenging even for experienced Excel & VBA developer, as I love to consider myself for one. Thus, today I was fighting for quite some time to add new columns to a dataframe,…
Tagged with: adding columns, based on columns, based on rows, computating column, dataframe, offset, Python, shift, shifting column
Python – *args, **kwargs and the normal ones (arg)
The good, the bad and the ugly is a famous western. In Python, “the ugly” is probably the **kwargs, as far as it has two stars before it. Anyhow, the idea of *args, **kwargs and arg is simply to pass arguments.…
Python – List Comprehension
List comprehension in Python is creation of lists from iterables and other lists, following a condition. It is lightning fast and it looks nice, because it is a 1 liner. In this article, I will show a few examples of…
Tagged with: conditions, dictionary comprehension, examples, list comprehension, matrix comprehension, Python
Python – What is the idea of “-1” in numpy reshape?
Today I was “fighting” with the numpy reshape function and I found something strange like matrix.reshape(1, -1). Thus, I have researched quite some time and I found, that the -1 is actually a placeholder for python to change the dimensions of…
Python – How to write string into a matrix. Printing a matrix.
Writing string into a matrix seems like a non-interesting and somehow trivial task. Anyway, manipulating with matrices and strings in Python is actually somehow interesting, thus I have decided to present a small algorithmic challenge. The task is pretty much…
Python – UnitTests in Jupyter Notebook
Unit testing is the “sweet” part of the programming – once you have tested “all the things”, you feel that the code you have submitted somewhere is actually valid and somehow it works. Furthermore, it allows you to nicely brag…
Python – Statsmodels Example – Video
Statsmodels is a nice statistics library in Python, which eases the data processing and analysis with Python. The library is available here – statsmodels.org and on its official web site the installation guide is always up to date. In the…
Excel – How to build magic square with VBA, following Python Mod Mathematics
Some 4 years ago I wrote an article for calculating magic square with python. What the article did was actually checking whether the square is magic, and not calculating a new one. This one is going to show how these…
Python – Plotting DataFrame and Working with Pandas – Video
Working with Pandas is like working with Excel on steroids – it can really do a lot of things fast, but somehow doing the easy things get complicated. In this video and article tutorial I am presenting loading data to…
Cryptography in Python
After writing about hashing passwords in Python, I have decided to publish the other parts of the chapter, “Cryptography” from the “Modern Python Standard Library Cookbook“, written by Alessandro Molina and and kindly provided to me by PacktPublishing.In the article,…
Tagged with: asking for password, compute_signature, cryptography, getpass, hashlib, history, hmac, ps, Python, verify a message integrity, verifying files integrity
Python – Platforms Jumping – CodeForces
Greedy algorithm is described as an algorithm that follows the problem-solving heuristic of making the locally optimal choice at each stage with the intent of finding a global optimum. Thus, it is always interesting to find a problem, which can…
Python Slice Notation
Python Slice Notation is somehow easy and tough in the same way. Lately, I was doing some competitive programming with Python and I have decided to write a summary of the slice notation, thus there is always a place for…
Python – Binary String Minimizing Problem – CodeForces
Codeforces has introduced a third division in their competitions. Initially I though this is the place to go for “easy” stuff, as far as some of the 2nd division problems require some advanced algorithms, like combinations of graphs and months…
Find all list (li) elements in a special position of a web site
So, my idea was to obtain all the elements, tagged with “li” from here – vitoshacademy.com/all, using BeautifulSoup4 and Python. Initially, I thought about running a simple soup.findAll("li") , but it also added the text from the menus, which were…
Scraping a web site up to the N-th level with Python
Scraping data from a web site has always been a pleasure with Python. If you come from the VBA world, the BeautifulSoup4 is indeed beautiful! I have been scraping a web site with python before, but now as I have…
Prime numbers matrix and coloring with Python and xlsxwriter
About 4 years ago, I wrote an article about prime numbers in Excel and their visualisation with VBA – vba-function-in-excel-to-check-prime-numbers. Pretty much, it displays the prime numbers in a 10 x N matrix, starting from 1 and finishing at N:…
Tagged with: align, bg_color, center, Excel, first column and first row of Excel is with index 0, font_color, format, matrix, prime numbers, Python, xlsxwriter
Scraping web site and writing its links to a database with Python
Scraping a web site with BeautifulSoup 4 and Python is a walk in the park, if you have scraped a web site with VBA before. Still, there are a few tricks, that should be taken into account. Trick #1 –…
Adding data to SqlLite3 database with Python
After reading the database to python from the previous post here, it is time to do the ADD data to the database. Our database is created like this:
|
1 2 3 4 5 |
CREATE TABLE "drummers" ( "Id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, "Name" TEXT, "Band" TEXT ) |
The database has autoincremented primary key, thus we do not need…
Connecting SqlLite3 database to Python and Reading it
Connecting SqlLite3 database to a Python file is actually 1 line of code: conn = sqlite3.connect(r"C:\Users\database.db") I have used a database from sqllitetutorial.net, which was rather neat. Then DB Browser for SQLLite a LEFT JOIN was implemented, in order to unite the…
Adding data to SqlLite3 database, but only one value with Python
Life is hard, when you are learning Python and you see things that are making no sense the way they run. After writing about adding data to SqlLite3 with Python, today I have tried to do the same, but for…
Find all HTML tags in a web page and print them from a sorted dictionary
Finding all HTML tags from a web page and recording these to a dictionary is magically easy task with Python (if you compare it with #VBA), when Beautiful Soup 4 is used. soup.find_all(True) loops through the tags and a simple…
Write Formulas in Excel through Python XlsWriter
Writing formulas with Excel with Python is a technique, that needs some practice, actually. Although XlsWriter has some good documentaion, not a lot is covered on the point, that concerns creating a file with a formula in it. First and…
Writing to a text file in Python from a variable, ignoring encoding problems
Writing to a file in Python is usually a simple 2-liner, which is easy to be written:
|
1 2 |
with io.open("soup.txt", "w") as file: file.write(soup_written) |
However, problems with the encoding may cause some research needed. Concerning the encoding this is a great article from the creator of…
Show all links from a website with Python and BeautifulSoup – Video
Showing all links from a website with Python and BeautifulSoup is actually 6 lines of code – 2 to define the used libraries and 4 to do the job:
|
1 2 3 4 5 6 7 8 |
import urllib.request from bs4 import BeautifulSoup resp = urllib.request.urlopen("https://www.vitoshacademy.com") soup = BeautifulSoup(resp, from_encoding=resp.info().get_param('charset')) for link in soup.find_all('a', href=True): print(link['href']) |
In general, BeautifulSoup comes out of the box with beautiful…
Tagged with: BeautifulSoup4, display links, extract links to file, print links, Python, show all links
Python – The birthday paradox algorithm
The birthday paradox is an interesting problem, mainly because of its somehow “unexpected” results. The problem usually is stated as the following: What is the minimum number of people in a room, in order to have a probability higher than…
Sets in Python – Differece, Symetric Difference, Union, Intersection
After writing the article scraping a web site up to the N-th level with Python, I have decided to summarize what could be achieved with Set() in this language. Pretty much, sets can be described as lists with a few…
Numpy and Pandas – Small introduction with Jupyter Notebook
Last week I was participating at the Python and ML Summit in Berlin. One of the most interesting lectures, on which I participated, was a workshop by, named “Reading all yourself was yesterday – How to turn large amounts of…
Top