Ubuntu on a Macbook Pro Mid 2019 (MacBookPro15,1)

Posted on 15 February 2022 in Technology • Tagged with apple, linux

Apple's T2 Security Chip gets itself in the way as much as possible when attempting to install Linux on a Mid 2019 Macbook Pro and a lot of the available documentation for Linux-on-the-mac online is pre-T2. After quite a bit of back and forth and eventually finding the right resources I managed to get Ubuntu installed on my MacBook Pro with everything working... except the microphone.

Initial Success and Failure

Getting started with this process was a bit of a minefield. The first resource I came across was How to Install and Dual-Boot Linux and macOS, an extremely detailed and useful documentation piece with one minor caveat (that I didn't take heed of initially):

As of right now, you cannot install Linux on the internal SSD of a newer MacBook Pro or Mac Pro (2018 or later). You can still install on an external drive, however.

Despite the warning I did manage to eventually cobble together a working base Ubuntu install using Ubuntu 20.04, a bootable USB-C drive (notably: not a USB-A adapter), and rEFInd. In the end I had a functional Ubuntu installation on the same internal SSD as my macOS install but none of the peripherals (keyboard …


Continue reading

Pip, Pis, Pandas and Wheels

Posted on 07 May 2018 in Technology • Tagged with arm, baby buddy, pip, pipenv, python, raspberry pi, troubleshooting

A user attempting to install Baby Buddy submitted an interesting issue with the following error during the pipenv install process:

THESE PACKAGES DO NOT MATCH THE HASHES FROM Pipfile.lock!. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them.
    docopt==0.6.2 from https://www.piwheels.org/simple/docopt/docopt-0.6.2-py2.py3-none-any.whl#sha256=0340515c74203895f92f87702896e45424bf51dc71bf15b4748450f50be04346 (from -r /tmp/pipenv-vf5_eub9-requirements/pipenv-k7_dvsro-requirement.txt (line 1)):
        Expected sha256 49b3a825280bd66b3aa83585ef59c4a8c82f2c8a522dbe754a8bc8d08c85c491
             Got        0340515c74203895f92f87702896e45424bf51dc71bf15b4748450f50be04346

Hash checking and Pipfile.lock are a part of the pipenv toolchain and meant to verify the integrity of packages being installed. Committing the lock file is recommended practice and generally something I have not had many problems with. There are some old tickets on GitHub reporting issues with this hashing between operating systems, but the latest versions of pipenv supposedly do not have these problems.

Why is this user getting a hash match error? I had a Pi lying around, so I decided to try replicating the issue. Many hours later, I got Baby Buddy up and running on my (second) Pi and learned a lot about the Python packaging process and how …


Continue reading

Advent of Code 2017: Days 16 - 20

Posted on 20 December 2017 in Technology • Tagged with advent of code, python

This post is part of the series, Advent of Code 2017, where I work my way through (part of) the 2017 Advent of Code challenges in order to re-learn some of the basics of Python and reflect on some simple concepts and functionality. All my challenge code is available in cdubz/advent-of-code-2017 on GitHub.

Day 16: One Billion Permutations in 0.535 Seconds

Ok, not really (: This challenge involved making specific modifications to a list 1,000,000,000 (one billion) times. Out of curiosity, the first thing I did was set up a progress bar loop to run the actual modifications all one billion times. The resulting progress bar estimated that the entire operation would take around 133 days. So... a different approach:

0
1
2
3
4
5
6
7
positions = [...]
movements = [...]
possibilities = [positions]
while True:
    positions = move(positions, movements)
    if positions in possibilities:
        break
    possibilities.append(positions)

Because the movements are always the same, eventually the positions list elements will return to their initial state. In this case, that happened on the 41st movement, making for a possibilities list of size 42. Armed with the answer to the ultimate …


Continue reading

Advent of Code 2017: Days 11 - 15

Posted on 15 December 2017 in Technology • Tagged with advent of code, python

This post is part of the series, Advent of Code 2017, where I work my way through (part of) the 2017 Advent of Code challenges in order to re-learn some of the basics of Python and reflect on some simple concepts and functionality. All my challenge code is available in cdubz/advent-of-code-2017 on GitHub.

Day 11: Navigating a Hexagon Grid

I fought with how to create and navigate a hexagon grid for a bit before I stumbled on an extremely useful post by Chris Schetter: Hexagon grids: coordinate systems and distance calculations. Working in 2d grids (with four possible movement directions) in Python is fairly easy with some dictionaries, but a hexagon grid means there are six potential directions and simple calculations of x,y coordinates won't quite get the job done. What Chris explains so well is a simple concept: flip the grid and add a z-axis.

For my solution, I "flip" the grid by turning it to the right twice such that north and south become east and west:

  \ n  /
nw +--+ ne              sw +--+ nw
  /    \                \ /    \ /
-+      +-    ---->    s +      + n
  \    /                / \    / \
sw +--+ se              se +--+ ne
  / s  \

This puts north and south along the x-axis instead …


Continue reading

Advent of Code 2017: Days 6 - 10

Posted on 10 December 2017 in Technology • Tagged with advent of code, python

This post is part of the series, Advent of Code 2017, where I work my way through (part of) the 2017 Advent of Code challenges in order to re-learn some of the basics of Python and reflect on some simple concepts and functionality. All my challenge code is available in cdubz/advent-of-code-2017 on GitHub.

Day 6: max(x, key=y)

Another simple revelation on built-ins from this challenge: the key argument can be used to modify what is evaluated by the max function. This concept is explained in detail as part of the accepted answer to this Stack Overflow post: python max function using 'key' and lambda expression.

For Python dictionaries, this means that the get() method can be used to return the key of the maximum value in a dictionary. This works because the max function sends each of the dict keys to the dict.get() method and evaluates that result instead of the key itself. Without this argument, max will actually evaluate the dictionary's keys, which isn't terribly useful:

0
1
2
3
4
>>> d = {0: 1, 2: 5, 3: 4}
>>> max(d)
3  # This is the *maximum value of …

Continue reading

Advent of Code 2017: Days 1 - 5

Posted on 05 December 2017 in Technology • Tagged with advent of code, python

This post is part of the series, Advent of Code 2017, where I work my way through (part of) the 2017 Advent of Code challenges in order to re-learn some of the basics of Python and reflect on some simple concepts and functionality. All my challenge code is available in cdubz/advent-of-code-2017 on GitHub.

Day 1: zip()

This challenge taught me about Python's built-in zip function. The basic goal is to compare values in a list with specific positional relationships to other values (e.g. next to, X steps from, etc.) in the same list. zip assists with this task by combining multiple lists in to tuples. My initial solution used a construct similar to:

0
1
2
3
4
digits = [1, 2, 3, 4, 5]
total = 0
for a, b in zip(digits[::1], digits[1::1]):
    if a == b:
        total += a

This code uses zip and Python's list indexing to evaluate each element in the digits list against the next element in the list by creating and combining two lists: one starting from the first digit in digits and one starting from the second digital in digits. Here are what …


Continue reading

Advent of Code 2017

Posted on 30 November 2017 in Technology • Tagged with advent of code, python

I taught myself Python a few years ago by following the wonderful Learn Python the Hard Way (LPTHW) series by Zed A. Shaw. Since then, I have spent a decent amount of time in Python largely in the Django framework. Django is a lot of fun to work with because it abstracts away much of the complexities of developing a full-featured web application in Python. In this way, however, it has also led me to forget some of the basic Python that I learned through LPTHW.

In order to recapture some of those early lessons (and maybe learn a few more Python 3 specific ones), I worked through (part of) the 2017 Advent of Code, a 25-day, language agnostic programming challenge series developed by Eric Watsl. I originally thought about using the series to learn a new language, but eventually realized that I still have a long way to go in Python.

This series explores my (often very basic) revelations and lessons learned while completing 20 of the 25 days of challenges (vacation travel cut the series short for me). The code I used for each day is available on GitHub. Also check out the advent-of-code and advent-of-code-2017 GitHub topics …


Continue reading

Consistent Selenium Testing in Python

Posted on 01 September 2017 in Technology • Tagged with django, python, saucelabs, selenium, testing, timestrap

Back in April, I learned about Timestrap, a self-hostable, Django-based time-tracking project from a post on HackerNews by Isaac Bythewood. As I have been learning Python in the past year or so, I reached out to Isaac and started contributing to the project. After getting familiar with the core application, I turned my attention to testing and eventually found my way to Selenium, a collection of browser automation tools used for frontend testing.

I had never worked with Selenium or other automated testing products, so it struck me as a great opportunity to get my feet wet in something new. After getting things up and running, we quickly learned that the test results were quite inconsistent across development environments - even to a point that occasionally tests would succeed when run individually, but fail with the full test case.

After much trial and error, we have settled on a (mostly) consistent setup for testing with Selenium, Python and SauceLabs. This produces much better results than testing in development environments and crossing fingers during CI. Hopefully this primer will help others facing similar challenges (as we had a lot of trouble finding good material on the subject).


Continue reading

RDAP Explorer

Posted on 06 February 2017 in Technology • Tagged with django, ip, ipv4, ipv6, ipwhois, nginx, python, rdap, uwsgi, whois

Having fallen behind a bit on Takeout Inspector, the 12 Years of Gmail series and some other projects, I decided to try to put something very simple together from beginning to end and actually launch it. One of my previous posts, Examining the Remnants of a Small DDoS Attack introduced me to the Python package ipwhois and the alternative WHOIS system RDAP. This eventually led me to a quick and simple project called RDAP Explorer...

What is RDAP?

According to APNIC

The Registration Data Access Protocol (RDAP) is an alternative to WHOIS for accessing Internet resource registration data. RDAP is designed to address a number of shortcomings in the existing Whois service. The most important changes are:

  • Standardization of queries and responses
  • Internationalization considerations to cater for languages other than English in data objects
  • Redirection capabilities to allow seamless referrals to other registries

The most important advantage of RDAP over WHOIS is the Standardization of queries and responses. While reviewing a large set of IP addresses, I found it rather difficult to deal with non-standard (and sometimes nonsensical) output of WHOIS queries. Mostly they were easy enough to parse, but the odd balls made the process annoying and time consuming …


Continue reading

12 Years of Gmail, Part 5: Mail

Posted on 05 December 2016 in Technology • Tagged with 12 years of gmail, email, graphing, plotly, python, takeout inspector, wordcloud

This post is part of my series, 12 Years of Gmail, taking a look at the data Google has accumulated on me over the past 12 years of using various Google services and documenting the learning experience developing an open source Python project (Takeout Inspector) to analyze that data.

After taking a look at the chat data in my export, I am finally ready to move on to some of the actual mail! Much of what I will look at here is pretty similar to what I was able to turn up with chat data. I tried to branch out a bit, bringing in a new package to create word clouds, and also refactored some of the Takeout Inspector code to form the beginning of a more "formal" report generating process (instead of just spitting out a single HTML file with only a certain subset of the data). Hopefully I can continue to improve this to a point allowing for easier report generation for any user. Anyway, on to the mail data!

Top …


Continue reading