python – matplotlib: drawing lines between points ignoring missing data-ThrowExceptions

Exception or error:

I have a set of data which I want plotted as a line-graph. For each series, some data is missing (but different for each series). Currently matplotlib does not draw lines which skip missing data: for example

import matplotlib.pyplot as plt

xs = range(8)
series1 = [1, 3, 3, None, None, 5, 8, 9]
series2 = [2, None, 5, None, 4, None, 3, 2]

plt.plot(xs, series1, linestyle='-', marker='o')
plt.plot(xs, series2, linestyle='-', marker='o')

plt.show()

results in a plot with gaps in the lines. How can I tell matplotlib to draw lines through the gaps? (I’d rather not have to interpolate the data).

How to solve:

You can mask the NaN values this way:

import numpy as np
import matplotlib.pyplot as plt

xs = np.arange(8)
series1 = np.array([1, 3, 3, None, None, 5, 8, 9]).astype(np.double)
s1mask = np.isfinite(series1)
series2 = np.array([2, None, 5, None, 4, None, 3, 2]).astype(np.double)
s2mask = np.isfinite(series2)

plt.plot(xs[s1mask], series1[s1mask], linestyle='-', marker='o')
plt.plot(xs[s2mask], series2[s2mask], linestyle='-', marker='o')

plt.show()

This leads to

Plot

Answer:

Qouting @Rutger Kassies (link) :

Matplotlib only draws a line between consecutive (valid) data points,
and leaves a gap at NaN values.

A solution if you are using Pandas, :

#pd.Series 
s.dropna().plot() #masking (as @Thorsten Kranz suggestion)

#pd.DataFrame
df['a_col_ffill'] = df['a_col'].ffill(method='ffill')
df['b_col_ffill'] = df['b_col'].ffill(method='ffill')  # changed from a to b
df[['a_col_ffill','b_col_ffill']].plot()

Answer:

A solution with pandas:

import matplotlib.pyplot as plt
import pandas as pd

def splitSerToArr(ser):
    return [ser.index, ser.as_matrix()]


xs = range(8)
series1 = [1, 3, 3, None, None, 5, 8, 9]
series2 = [2, None, 5, None, 4, None, 3, 2]

s1 = pd.Series(series1, index=xs)
s2 = pd.Series(series2, index=xs)

plt.plot( *splitSerToArr(s1.dropna()), linestyle='-', marker='o')
plt.plot( *splitSerToArr(s2.dropna()), linestyle='-', marker='o')

plt.show()

The splitSerToArr function is very handy, when plotting in Pandas. This is the output:enter image description here

Answer:

Without interpolation you’ll need to remove the None’s from the data. This also means you’ll need to remove the X-values corresponding to None’s in the series. Here’s an (ugly) one liner for doing that:

  x1Clean,series1Clean = zip(* filter( lambda x: x[1] is not None , zip(xs,series1) ))

The lambda function returns False for None values, filtering the x,series pairs from the list, it then re-zips the data back into its original form.

Answer:

For what it may be worth, after some trial and error I would like to add one clarification to Thorsten’s solution. Hopefully saving time for users who looked elsewhere after having tried this approach.

I was unable to get success with an identical problem while using

from pyplot import *

and attempting to plot with

plot(abscissa[mask],ordinate[mask])

It seemed it was required to use import matplotlib.pyplot as plt to get the proper NaNs handling, though I cannot say why.

Answer:

Another solution for pandas DataFrames:

plot = df.plot(style='o-') # draw the lines so they appears in the legend
colors = [line.get_color() for line in plot.lines] # get the colors of the markers
df = df.interpolate(limit_area='inside') # interpolate
lines = plot.plot(df.index, df.values) # add more lines (with a new set of colors)
for color, line in zip(colors, lines):
  line.set_color(color) # overwrite the new lines colors with the same colors as the old lines

Answer:

Perhaps I missed the point, but I believe Pandas now does this automatically. The example below is a little involved, and requires internet access, but the line for China has lots of gaps in the early years, hence the straight line segments.

import pandas as pd 
import numpy as np 
import matplotlib.pyplot as plt

# read data from Maddison project 
url = 'http://www.ggdc.net/maddison/maddison-project/data/mpd_2013-01.xlsx'
mpd = pd.read_excel(url, skiprows=2, index_col=0, na_values=[' ']) 
mpd.columns = map(str.rstrip, mpd.columns)

# select countries 
countries = ['England/GB/UK', 'USA', 'Japan', 'China', 'India', 'Argentina']
mpd = mpd[countries].dropna()
mpd = mpd.rename(columns={'England/GB/UK': 'UK'})
mpd = np.log(mpd)/np.log(2)  # convert to log2 

# plots
ax = mpd.plot(lw=2)
ax.set_title('GDP per person', fontsize=14, loc='left')
ax.set_ylabel('GDP Per Capita (1990 USD, log2 scale)')
ax.legend(loc='upper left', fontsize=10, handlelength=2, labelspacing=0.15)
fig = ax.get_figure()
fig.show() 

Leave a Reply

Your email address will not be published. Required fields are marked *