AW — notes & thoughts

Drawing a coffee plot, with LLMs ep0

Sat, 10 May 2025 21:28:43 -0500

LLM-assisted work?

Will the LLM take over us? Yes, no. Maybe.

The only thing I’m sure about is that those cool stuff are not able to assess the certainty of text (“opinions?”) they generate.

Moreover, In my humanly-limited experience, I have yet to see LLMs to meaningful contribute to any asset, i.e. to add valuable and extensive contribution to non-throaway code. All of this may seem strong, but hey, this does not mean productivity cannot be positively impacted.

Sparse notes

Wed, 01 Mar 2023 21:28:43 -0500

Data science

march Effect of irrelevant features - a nice experimental test
march Matrix factorizaton recap in 6minutes
march mean encodings: a comprehesive study, check also category_encoders sklearn library
march relationship between SVD and PCA + a comprehensive tutorial on arxiv
march Naive-Bayes classification from scratch
On Embeddings for Numerical Features in Tabular Deep Learning
Euclidean vs cosine distance

Data viz

Complimentary

Temporary

Staging area for links that will be forgotten soon:

march H&M personalized fashion reccomendation
march learning D3js + essential training + custom visuals into powerbi

Q1 2023: Miscellanea (playlist)

Wed, 01 Feb 2023 21:28:43 -0500

Usare i numeri ciecamente: la fallacia di McNamara

Due note brevi sulla fallacia di McNamara (segretario della difesa USA durante la guerra in Vietnam), spesso riassunta come “fallacia quantitativa”

“the facts which count best count most". [I fatti che si contano meglio sono quelli che contano di più]

Spesso è figlia delle organizzazioni, ma colpisce (duro) individui e decisioni. Come fare a riprodurla? Ecco un tutorial (preso da qui):

As reported by Smith, Yankelovich describes the fallacy as follows:

What's all that for? Thinking of the BTC energy consumptions

Tue, 10 Jan 2023 21:28:43 -0500

I can remember the snow

Born on the cusp of the 90s, I was privileged to witness a period of great optimism and progress. The winds of freedom and democracy seemed to be blowing strong, and economies were thriving. One particularly noteworthy achievement from this time was the establishment of the Montreal Protocol, a treaty designed to phase out ozone-depleting substances, a move that has proven highly successful.

I have fond memories of being a child during this time, including TV commercials¹ that highlighted the importance of protecting polar bears. Even after all these years, the imagery of these ads stays with me, and their message continues to resonate, a testament to their powerful impact.

A (reasonably) complex radar chart

Fri, 11 Mar 2022 21:28:43 -0500

A radar (or spider) chart is a convenient method of displaying multivariate data in a form of 2-dimensional chart.

When I say convenient, I mean it is particularly well-accepted by people and gives a good intuition when comparing different items/subjects.

Think about comparing 2 (or more) football players in terms of dribbling, running speed, shots and pressure. Even with only four variables bar plot would become cumbersome to deal with and not immediate to perceive. That’s why they are particularly popular in sports analytics.

The economics of the agri-food value chain

Tue, 01 Mar 2022 21:28:43 -0500

Course review

Let me paraphrase what Tukey once said: I truly believe that getting in contact with different fields and expertise is one of the perks of being a data scientist. Pictorially:

The best thing about being a ~~statistician~~ data scientist, Mr Tukey once told a colleague, is that you get to play in everyone’s backyard.

So, even if you can legitimately play in everyone’s backyard - which oftentimes means cleaning it as a first step, it also means that you will be speaking to people with different backgrounds ad expertise.

Google 20 for 1 stock split

Wed, 02 Feb 2022 21:28:43 -0500

Google announced Tuesday its results for Q4 2021 and that they plan to split shares at 20 for 1 this Tuesday. Basically, if you are a shareholder, for each 1 share of Google at (about) 3000$ dollars you will have (at the ex-date) 20 shares at 150$.

Nothing should really change, but there might be some financial effects, for instance, due to the inclusion of the stock in new indices or increased market liquidity.

Crescita Economica

Sun, 31 Jan 2021 21:28:43 -0500

Questa pagina traduce liberamente e parzialmente, l’articolo di Max Roser (secondo licenza) (2013) - “Economic Growth”. Published online at OurWorldInData

Riferitevi alla versione originale per le citazioni e per il testo completo, io mi sono permesso di aggiungere qualche nota interessante per noi italiani.

Con crescita economica descriviamo un aumento nella qualità a quantità dei beni e servizi economici che una società produce o consuma.

Mentre la definizione di crescita economica è chiara, misurare la crescita in sè risulta estremamente difficile. La crescita è spesso misurata tramite un aumento nel reddito delle famiglie o del PIL (aggiustato per l’inflazione), ma è importante ricordare che queste misure non sono la definizione di crescita - proprio come l’aspettativa di vita è una misura della “salute di una popolazione” ma certamente non coincide con la definizione stessa. Le misure legate al reddito sono solamente un modo di capire la diseguaglianza tra paesi e la variazione di ricchezza nel tempo.

Short notes on namedtuples, NamedTuples and Data classes

Mon, 11 Jan 2021 21:28:43 -0500

Both dataclasses and tuples are based on the attrs project, the one Python Library everybody needs and are fast object types (factory methods) designed to simplify and reduce code.

namedtuple and NamedTuple

Both immutable
Tuple-based (hence fast) but far better than tuples
NamedTuple is the typed version of namedtuple
immutable, iterable, hashable, unpackable
backward-compatible with tuple (e.g you can access a namedtuple by index)
default arguments supported from Python 3.7+
fast! C-based
example ¹

Point = namedtuple('Point', 'x y')
pt1 = Point(1.0, 5.0)
pt2 = Point(2.5, 1.5)

from math import sqrt
# use index referencing
line_length = sqrt((pt1[0]-pt2[0])**2 + (pt1[1]-pt2[1])**2)
 # use tuple unpacking
x1, y1 = pt1

Dataclasses

Are mutable
post-init processing can be used to create fields depending on other fields or even to perform input validation²
all implementation is written in Python, so slower wrt tuple-based data types
Validation of types at runtime not supported natively (or cumbersome) but easy with decorator @enforce_typing
from Python 3.7
are just regular Classes (e.g. inheritance) withot writing boilerplate code
inappropriate when API compatibility with tuples or dicts is requested
bonus PyCon talk, if you have time
example³

from dataclasses import dataclass

@dataclass(unsafe_hash=True)
class InventoryItem:
 '''Class for keeping track of an item in inventory.'''
 name: str
 unit_price: float
 quantity_on_hand: int = 0

 def total_cost(self) -> float:
 return self.unit_price * self.quantity_on_hand

Other fancy data types (not from the stdlib)

pydantic: enforces type hints at runtime providing user-readable errors when data is invalid. Seems relatively popular.

# sample input val via Regexp
from dataclasses import dataclass
import re
@dataclass
class Widget:
 id: int
 def __post_init__(self):
 id_condition = re.match(r"[0-9]{4}", str(self.id))
 if not id_condition:
 print(f"{self.id} doesn't follow pattern [0-9]{{4}}")
 raise CustomException

brief explanation on stackoverlow ↩︎

Add a row to pandas' DataFrame

Sat, 31 Oct 2020 21:28:43 -0500

Quite often, it is needed to fill or modify dataframes with data that gets computed at runtime. This post draws heavily from the stackoverlow question “Add one row to pandas DataFrame”. Here I show the 4 methods that were proposed to append the data and discuss them. A 5th “improper” method operating on columns is added just for sake of comparison with respect to the most efficient option.

If you want just the takeaways: