Migrations

Some tips to switch from Pandas 1.x to 2.1.1, with SqlAlchemy 2.0.21 and Python 3.9 (and maybe Python 3.11)

ImportError: cannot import name SettingWithCopyWarning from pandas.core.common

It can happens if you use:

import warnings
from pandas.core.common import SettingWithCopyWarning
warnings.simplefilter(action='ignore', category=SettingWithCopyWarning)

Now prefer:

import pandas as pd
...
pd.set_option('mode.chained_assignment', None)

And if you script is loaded from several Python versions, you can do:

import pandas as pd
...
pandas_version = pd.__version__
 
if pandas_version >= '2.1.1':
    pd.set_option('mode.chained_assignment', None)
 
else:
    import warnings
    from pandas.core.common import SettingWithCopyWarning
    warnings.simplefilter(action='ignore', category=SettingWithCopyWarning)

sqlalchemy.exc.ObjectNotExecutableError: Not an executable object

It can happens when you execute a query from a string, example:

from sqlalchemy import create_engine
from _params import my_username, my_password, my_host, my_port, my_database
 
my_engine = create_engine('mysql+mysqldb://%s:%s@%s:%i/%s?charset=utf8mb4' % (my_username, my_password, my_host, my_port, my_database))
 
my_connection = my_engine.connect()
 
my_query = my_connection.execute('SELECT 1 FROM mytable ;')

Just put your SQL query in text:

from sqlalchemy import create_engine, text
from _params import my_username, my_password, my_host, my_port, my_database
 
my_engine = create_engine('mysql+mysqldb://%s:%s@%s:%i/%s?charset=utf8mb4' % (my_username, my_password, my_host, my_port, my_database))
 
my_connection = my_engine.connect()
 
my_query = my_connection.execute(text('SELECT 1 FROM mytable ;'))

Specify dtype option on import or set low_memory=False

Specify low_memory when you read your file:

df = pd.read_csv('your file.csv', low_memory=False)

FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated

It can happens if you do a pd.concat, something like:

df_1 = ...
df_2 = ...
df_3 = ...
 
df = pd.concat([df_1, df_2, df_3], axis=0)

Prefer that:

ListMyDfs = ['df_1', 'df_2', 'df_3']
 
df = pd.DataFrame()
 
for MyDf in ListMyDfs:
   if not globals()[MyDf].empty:
        df = pd.concat([df, globals()[MyDf]], axis=0)

Liens ou pièces jointes

France-Departements-Deformation.zip	[France-Departements-Deformation]	335 Ko
simple_countries.zip	[simple_countries]	1880 Ko