{ "metadata": { "name": "pandas_timeseries" }, "nbformat": 3, "nbformat_minor": 0, "worksheets": [ { "cells": [ { "cell_type": "heading", "level": 1, "metadata": {}, "source": [ "Timeseries with pandas" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Working with time-series data is an important part of data analysis.\n", "\n", "Starting with v0.8, the _pandas_ library has included a rich API for time-series manipulations.\n", "\n", "The _pandas_ time-series API includes:\n", "\n", "- Creating date ranges\n", " - From files\n", " - From scratch\n", "- Manipulations: Shift, resample, filter\n", "- Field accessors (e.g., hour of day)\n", "- Plotting\n", "- Time zones (localization and conversion)\n", "- Dual representations (point-in-time vs interval)" ] }, { "cell_type": "code", "collapsed": false, "input": [ "from datetime import datetime, date, time\n", "import sys\n", "sys.version" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 1, "text": [ "'2.7.3 |EPD 7.3-2 (64-bit)| (default, Apr 12 2012, 11:14:05) \\n[GCC 4.0.1 (Apple Inc. build 5493)]'" ] } ], "prompt_number": 1 }, { "cell_type": "code", "collapsed": false, "input": [ "import pandas as pd\n", "from pandas import Series, DataFrame, Panel\n", "pd.__version__" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 2, "text": [ "'0.9.1.dev-88c9ef5'" ] } ], "prompt_number": 2 }, { "cell_type": "code", "collapsed": false, "input": [ "np.__version__" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 3, "text": [ "'1.6.1'" ] } ], "prompt_number": 3 }, { "cell_type": "code", "collapsed": false, "input": [ "import matplotlib.pyplot as plt\n", "import matplotlib as mpl\n", "mpl.rc('figure', figsize=(10, 8))\n", "mpl.__version__" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 4, "text": [ "'1.1.0'" ] } ], "prompt_number": 4 }, { "cell_type": "markdown", "metadata": {}, "source": [ "###Example using tick data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Sample trade ticks from 2011-11-01 to 2011-11-03 for a single security" ] }, { "cell_type": "code", "collapsed": false, "input": [ "with open('data.csv', 'r') as fh:\n", " print fh.readline() # headers\n", " print fh.readline() # first row" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "Date,Time,Price,Volume,Exchange Code,Sales Condition,Correction Indicator,Sequence Number,Trade Stop Indicator,Source of Trade,MDS 127 / TRF (Trade Reporting Facility) (*),Exclude Record Flag,Filtered Price\r\n", "\n", "11/01/2011,08:04:56.364,104.55,100,P,T,00,196,N,C,,,\r\n", "\n" ] } ], "prompt_number": 5 }, { "cell_type": "markdown", "metadata": {}, "source": [ "`parse_dates`: use a list or dict for flexible (possibly multi-column) date parsing" ] }, { "cell_type": "code", "collapsed": false, "input": [ "data = pd.read_csv('data.csv', \n", " parse_dates={'Timestamp': ['Date', 'Time']},\n", " index_col='Timestamp')\n", "data" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 6, "text": [ "\n", "DatetimeIndex: 14632 entries, 2011-11-01 08:04:56.364000 to 2011-11-03 16:09:47.834000\n", "Data columns:\n", "Price 14632 non-null values\n", "Volume 14632 non-null values\n", "Exchange Code 14632 non-null values\n", "Sales Condition 14632 non-null values\n", "Correction Indicator 14632 non-null values\n", "Sequence Number 14632 non-null values\n", "Trade Stop Indicator 14632 non-null values\n", "Source of Trade 14632 non-null values\n", "MDS 127 / TRF (Trade Reporting Facility) (*) 2421 non-null values\n", "Exclude Record Flag 28 non-null values\n", "Filtered Price 0 non-null values\n", "dtypes: float64(2), int64(3), object(6)" ] } ], "prompt_number": 6 }, { "cell_type": "code", "collapsed": false, "input": [ "ticks = data.ix[:, ['Price', 'Volume']]\n", "ticks.head()" ], "language": "python", "metadata": {}, "outputs": [ { "html": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
PriceVolume
Timestamp
2011-11-01 08:04:56.364000 104.55 100
2011-11-01 08:53:07.459000 104.43 150
2011-11-01 08:53:09.284000 104.43 100
2011-11-01 08:54:03.921000 104.50 100
2011-11-01 08:58:06.162000 104.46 1000
\n", "
" ], "output_type": "pyout", "prompt_number": 7, "text": [ " Price Volume\n", "Timestamp \n", "2011-11-01 08:04:56.364000 104.55 100\n", "2011-11-01 08:53:07.459000 104.43 150\n", "2011-11-01 08:53:09.284000 104.43 100\n", "2011-11-01 08:54:03.921000 104.50 100\n", "2011-11-01 08:58:06.162000 104.46 1000" ] } ], "prompt_number": 7 }, { "cell_type": "markdown", "metadata": {}, "source": [ "####`resample`: regularization and frequency conversion" ] }, { "cell_type": "code", "collapsed": false, "input": [ "bars = ticks.Price.resample('1min', how='ohlc')\n", "bars" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 8, "text": [ "\n", "DatetimeIndex: 3366 entries, 2011-11-01 08:05:00 to 2011-11-03 16:10:00\n", "Freq: T\n", "Data columns:\n", "open 1104 non-null values\n", "high 1104 non-null values\n", "low 1104 non-null values\n", "close 1104 non-null values\n", "dtypes: float64(4)" ] } ], "prompt_number": 8 }, { "cell_type": "code", "collapsed": false, "input": [ "minute_range = bars.high - bars.low\n", "minute_range.describe()" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 9, "text": [ "count 1104.000000\n", "mean 0.017898\n", "std 0.018568\n", "min 0.000000\n", "25% 0.000875\n", "50% 0.013900\n", "75% 0.029000\n", "max 0.150000" ] } ], "prompt_number": 9 }, { "cell_type": "code", "collapsed": false, "input": [ "minute_return = bars.close / bars.open - 1\n", "minute_return.describe()" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 10, "text": [ "count 1104.000000\n", "mean 0.000003\n", "std 0.000191\n", "min -0.001442\n", "25% -0.000096\n", "50% 0.000000\n", "75% 0.000096\n", "max 0.000962" ] } ], "prompt_number": 10 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Compute a VWAP using resample" ] }, { "cell_type": "code", "collapsed": false, "input": [ "volume = ticks.Volume.resample('1min', how='sum')\n", "value = ticks.prod(axis=1).resample('1min', how='sum')\n", "vwap = value / volume" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 11 }, { "cell_type": "markdown", "metadata": {}, "source": [ "####Convenient indexing for time series data" ] }, { "cell_type": "code", "collapsed": false, "input": [ "vwap.ix['2011-11-01 09:27':'2011-11-01 09:32']" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 12, "text": [ "Timestamp\n", "2011-11-01 09:27:00 NaN\n", "2011-11-01 09:28:00 NaN\n", "2011-11-01 09:29:00 104.170000\n", "2011-11-01 09:30:00 103.300800\n", "2011-11-01 09:31:00 104.231368\n", "2011-11-01 09:32:00 104.236679\n", "Freq: T" ] } ], "prompt_number": 12 }, { "cell_type": "markdown", "metadata": {}, "source": [ "####`at_time`: same (b)at_time (same bat channel)" ] }, { "cell_type": "code", "collapsed": false, "input": [ "bars.open.at_time('9:30')" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 13, "text": [ "Timestamp\n", "2011-11-01 09:30:00 103.3008\n", "2011-11-02 09:30:00 103.8600\n", "2011-11-03 09:30:00 NaN\n", "Name: open" ] } ], "prompt_number": 13 }, { "cell_type": "code", "collapsed": false, "input": [ "bars.close.at_time('16:00')" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 14, "text": [ "Timestamp\n", "2011-11-01 16:00:00 104.56\n", "2011-11-02 16:00:00 104.32\n", "2011-11-03 16:00:00 103.82\n", "Name: close" ] } ], "prompt_number": 14 }, { "cell_type": "markdown", "metadata": {}, "source": [ "####`between_time`: intraday time range" ] }, { "cell_type": "code", "collapsed": false, "input": [ "filtered = vwap.between_time('10:00', '16:00')\n", "filtered.head(20)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 15, "text": [ "Timestamp\n", "2011-11-01 10:00:00 104.145000\n", "2011-11-01 10:01:00 104.245387\n", "2011-11-01 10:02:00 104.170771\n", "2011-11-01 10:03:00 104.170000\n", "2011-11-01 10:04:00 104.160952\n", "2011-11-01 10:05:00 104.110870\n", "2011-11-01 10:06:00 104.093576\n", "2011-11-01 10:07:00 104.145132\n", "2011-11-01 10:08:00 104.157924\n", "2011-11-01 10:09:00 104.154646\n", "2011-11-01 10:10:00 104.150756\n", "2011-11-01 10:11:00 104.182591\n", "2011-11-01 10:12:00 104.116803\n", "2011-11-01 10:13:00 104.092174\n", "2011-11-01 10:14:00 104.067768\n", "2011-11-01 10:15:00 104.070000\n", "2011-11-01 10:16:00 104.073333\n", "2011-11-01 10:17:00 104.080906\n", "2011-11-01 10:18:00 104.082566\n", "2011-11-01 10:19:00 104.085140" ] } ], "prompt_number": 15 }, { "cell_type": "code", "collapsed": false, "input": [ "vol = volume.between_time('10:00', '16:00')\n", "vol.head(20)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 16, "text": [ "Timestamp\n", "2011-11-01 10:00:00 200\n", "2011-11-01 10:01:00 18767\n", "2011-11-01 10:02:00 67287\n", "2011-11-01 10:03:00 2000\n", "2011-11-01 10:04:00 2100\n", "2011-11-01 10:05:00 2300\n", "2011-11-01 10:06:00 11509\n", "2011-11-01 10:07:00 3526\n", "2011-11-01 10:08:00 734\n", "2011-11-01 10:09:00 1106\n", "2011-11-01 10:10:00 1800\n", "2011-11-01 10:11:00 1760\n", "2011-11-01 10:12:00 2961\n", "2011-11-01 10:13:00 2300\n", "2011-11-01 10:14:00 5818\n", "2011-11-01 10:15:00 200\n", "2011-11-01 10:16:00 600\n", "2011-11-01 10:17:00 1040\n", "2011-11-01 10:18:00 1576\n", "2011-11-01 10:19:00 2813" ] } ], "prompt_number": 16 }, { "cell_type": "markdown", "metadata": {}, "source": [ "####`fillna`: handling missing data" ] }, { "cell_type": "code", "collapsed": false, "input": [ "filtered.ix['2011-11-03':'2011-11-04'].head(20)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 17, "text": [ "Timestamp\n", "2011-11-03 10:00:00 NaN\n", "2011-11-03 10:01:00 103.968667\n", "2011-11-03 10:02:00 NaN\n", "2011-11-03 10:03:00 103.997500\n", "2011-11-03 10:04:00 104.044295\n", "2011-11-03 10:05:00 NaN\n", "2011-11-03 10:06:00 104.135118\n", "2011-11-03 10:07:00 104.130000\n", "2011-11-03 10:08:00 104.120000\n", "2011-11-03 10:09:00 104.101250\n", "2011-11-03 10:10:00 NaN\n", "2011-11-03 10:11:00 NaN\n", "2011-11-03 10:12:00 NaN\n", "2011-11-03 10:13:00 NaN\n", "2011-11-03 10:14:00 104.086667\n", "2011-11-03 10:15:00 104.049200\n", "2011-11-03 10:16:00 104.028333\n", "2011-11-03 10:17:00 103.976438\n", "2011-11-03 10:18:00 103.912492\n", "2011-11-03 10:19:00 NaN" ] } ], "prompt_number": 17 }, { "cell_type": "code", "collapsed": false, "input": [ "filled = filtered.fillna(method='pad', limit=1)\n", "filled.ix['2011-11-03':'2011-11-04'].head(20)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 18, "text": [ "Timestamp\n", "2011-11-03 10:00:00 104.305099\n", "2011-11-03 10:01:00 103.968667\n", "2011-11-03 10:02:00 103.968667\n", "2011-11-03 10:03:00 103.997500\n", "2011-11-03 10:04:00 104.044295\n", "2011-11-03 10:05:00 104.044295\n", "2011-11-03 10:06:00 104.135118\n", "2011-11-03 10:07:00 104.130000\n", "2011-11-03 10:08:00 104.120000\n", "2011-11-03 10:09:00 104.101250\n", "2011-11-03 10:10:00 104.101250\n", "2011-11-03 10:11:00 NaN\n", "2011-11-03 10:12:00 NaN\n", "2011-11-03 10:13:00 NaN\n", "2011-11-03 10:14:00 104.086667\n", "2011-11-03 10:15:00 104.049200\n", "2011-11-03 10:16:00 104.028333\n", "2011-11-03 10:17:00 103.976438\n", "2011-11-03 10:18:00 103.912492\n", "2011-11-03 10:19:00 103.912492" ] } ], "prompt_number": 18 }, { "cell_type": "code", "collapsed": false, "input": [ "vol = vol.fillna(0.)\n", "vol.head(20)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 19, "text": [ "Timestamp\n", "2011-11-01 10:00:00 200\n", "2011-11-01 10:01:00 18767\n", "2011-11-01 10:02:00 67287\n", "2011-11-01 10:03:00 2000\n", "2011-11-01 10:04:00 2100\n", "2011-11-01 10:05:00 2300\n", "2011-11-01 10:06:00 11509\n", "2011-11-01 10:07:00 3526\n", "2011-11-01 10:08:00 734\n", "2011-11-01 10:09:00 1106\n", "2011-11-01 10:10:00 1800\n", "2011-11-01 10:11:00 1760\n", "2011-11-01 10:12:00 2961\n", "2011-11-01 10:13:00 2300\n", "2011-11-01 10:14:00 5818\n", "2011-11-01 10:15:00 200\n", "2011-11-01 10:16:00 600\n", "2011-11-01 10:17:00 1040\n", "2011-11-01 10:18:00 1576\n", "2011-11-01 10:19:00 2813" ] } ], "prompt_number": 19 }, { "cell_type": "markdown", "metadata": {}, "source": [ "####Simple plotting" ] }, { "cell_type": "code", "collapsed": false, "input": [ "filled.ix['2011-11-03':'2011-11-04'].plot()\n", "plt.ylim(103.5, 104.5)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 20, "text": [ "(103.5, 104.5)" ] }, { "output_type": "display_data", "svg": [ "\n", "\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n" ], "text": [ "" ] } ], "prompt_number": 20 }, { "cell_type": "code", "collapsed": true, "input": [ "vwap.ix['2011-11-03':'2011-11-04'].plot()\n", "plt.ylim(103.5, 104.5)\n", "vol.ix['2011-11-03':'2011-11-04'].plot(secondary_y=True, style='r')" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 21, "text": [ "" ] }, { "output_type": "display_data", "svg": [ "\n", "\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n" ], "text": [ "" ] } ], "prompt_number": 21 }, { "cell_type": "markdown", "metadata": {}, "source": [ "####Lead/lag" ] }, { "cell_type": "code", "collapsed": false, "input": [ "ticks.head()" ], "language": "python", "metadata": {}, "outputs": [ { "html": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
PriceVolume
Timestamp
2011-11-01 08:04:56.364000 104.55 100
2011-11-01 08:53:07.459000 104.43 150
2011-11-01 08:53:09.284000 104.43 100
2011-11-01 08:54:03.921000 104.50 100
2011-11-01 08:58:06.162000 104.46 1000
\n", "
" ], "output_type": "pyout", "prompt_number": 22, "text": [ " Price Volume\n", "Timestamp \n", "2011-11-01 08:04:56.364000 104.55 100\n", "2011-11-01 08:53:07.459000 104.43 150\n", "2011-11-01 08:53:09.284000 104.43 100\n", "2011-11-01 08:54:03.921000 104.50 100\n", "2011-11-01 08:58:06.162000 104.46 1000" ] } ], "prompt_number": 22 }, { "cell_type": "markdown", "metadata": {}, "source": [ "**`shift`** realigns values" ] }, { "cell_type": "code", "collapsed": false, "input": [ "ticks.shift(1).head()" ], "language": "python", "metadata": {}, "outputs": [ { "html": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
PriceVolume
Timestamp
2011-11-01 08:04:56.364000 NaN NaN
2011-11-01 08:53:07.459000 104.55 100
2011-11-01 08:53:09.284000 104.43 150
2011-11-01 08:54:03.921000 104.43 100
2011-11-01 08:58:06.162000 104.50 100
\n", "
" ], "output_type": "pyout", "prompt_number": 23, "text": [ " Price Volume\n", "Timestamp \n", "2011-11-01 08:04:56.364000 NaN NaN\n", "2011-11-01 08:53:07.459000 104.55 100\n", "2011-11-01 08:53:09.284000 104.43 150\n", "2011-11-01 08:54:03.921000 104.43 100\n", "2011-11-01 08:58:06.162000 104.50 100" ] } ], "prompt_number": 23 }, { "cell_type": "code", "collapsed": false, "input": [ "ticks.shift(-1).head()" ], "language": "python", "metadata": {}, "outputs": [ { "html": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
PriceVolume
Timestamp
2011-11-01 08:04:56.364000 104.43 150
2011-11-01 08:53:07.459000 104.43 100
2011-11-01 08:53:09.284000 104.50 100
2011-11-01 08:54:03.921000 104.46 1000
2011-11-01 08:58:06.162000 104.46 1000
\n", "
" ], "output_type": "pyout", "prompt_number": 24, "text": [ " Price Volume\n", "Timestamp \n", "2011-11-01 08:04:56.364000 104.43 150\n", "2011-11-01 08:53:07.459000 104.43 100\n", "2011-11-01 08:53:09.284000 104.50 100\n", "2011-11-01 08:54:03.921000 104.46 1000\n", "2011-11-01 08:58:06.162000 104.46 1000" ] } ], "prompt_number": 24 }, { "cell_type": "markdown", "metadata": {}, "source": [ "**`tshift`** manipulates index values" ] }, { "cell_type": "code", "collapsed": false, "input": [ "ticks.tshift(1, 'min').head()" ], "language": "python", "metadata": {}, "outputs": [ { "html": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
PriceVolume
2011-11-01 08:05:56.364000 104.55 100
2011-11-01 08:54:07.459000 104.43 150
2011-11-01 08:54:09.284000 104.43 100
2011-11-01 08:55:03.921000 104.50 100
2011-11-01 08:59:06.162000 104.46 1000
\n", "
" ], "output_type": "pyout", "prompt_number": 25, "text": [ " Price Volume\n", "2011-11-01 08:05:56.364000 104.55 100\n", "2011-11-01 08:54:07.459000 104.43 150\n", "2011-11-01 08:54:09.284000 104.43 100\n", "2011-11-01 08:55:03.921000 104.50 100\n", "2011-11-01 08:59:06.162000 104.46 1000" ] } ], "prompt_number": 25 }, { "cell_type": "markdown", "metadata": {}, "source": [ "####SSS: stupidly simple strategy" ] }, { "cell_type": "code", "collapsed": false, "input": [ "minute_return.head()" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 26, "text": [ "Timestamp\n", "2011-11-01 08:05:00 0\n", "2011-11-01 08:06:00 NaN\n", "2011-11-01 08:07:00 NaN\n", "2011-11-01 08:08:00 NaN\n", "2011-11-01 08:09:00 NaN\n", "Freq: T" ] } ], "prompt_number": 26 }, { "cell_type": "code", "collapsed": false, "input": [ "mr = minute_return.between_time('9:30', '16:00')\n", "mr.head()" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 27, "text": [ "Timestamp\n", "2011-11-01 09:30:00 0.000000\n", "2011-11-01 09:31:00 0.000192\n", "2011-11-01 09:32:00 -0.000192\n", "2011-11-01 09:33:00 0.000000\n", "2011-11-01 09:34:00 -0.000096" ] } ], "prompt_number": 27 }, { "cell_type": "code", "collapsed": false, "input": [ "lagged = mr.shift(1)\n", "lagged.head()" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 28, "text": [ "Timestamp\n", "2011-11-01 09:30:00 NaN\n", "2011-11-01 09:31:00 0.000000\n", "2011-11-01 09:32:00 0.000192\n", "2011-11-01 09:33:00 -0.000192\n", "2011-11-01 09:34:00 0.000000" ] } ], "prompt_number": 28 }, { "cell_type": "markdown", "metadata": {}, "source": [ "We shouldn't use shift here because:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "lagged.at_time('9:30')" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 29, "text": [ "Timestamp\n", "2011-11-01 09:30:00 NaN\n", "2011-11-02 09:30:00 0.000191\n", "2011-11-03 09:30:00 0.000192" ] } ], "prompt_number": 29 }, { "cell_type": "code", "collapsed": false, "input": [ "mr.at_time('16:00')" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 30, "text": [ "Timestamp\n", "2011-11-01 16:00:00 0.000191\n", "2011-11-02 16:00:00 0.000192\n", "2011-11-03 16:00:00 -0.000193" ] } ], "prompt_number": 30 }, { "cell_type": "code", "collapsed": false, "input": [ "lagged = minute_return.tshift(1, 'min').between_time('9:30', '16:00')\n", "lagged.at_time('9:30')" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 31, "text": [ "Timestamp\n", "2011-11-01 09:30:00 0\n", "2011-11-02 09:30:00 0\n", "2011-11-03 09:30:00 NaN" ] } ], "prompt_number": 31 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's play" ] }, { "cell_type": "code", "collapsed": false, "input": [ "pd.ols(y=mr, x=lagged)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 32, "text": [ "\n", "-------------------------Summary of Regression Analysis-------------------------\n", "\n", "Formula: Y ~ + \n", "\n", "Number of Observations: 1009\n", "Number of Degrees of Freedom: 2\n", "\n", "R-squared: 0.0011\n", "Adj R-squared: 0.0001\n", "\n", "Rmse: 0.0002\n", "\n", "F-stat (1, 1007): 1.1007, p-value: 0.2944\n", "\n", "Degrees of Freedom: model 1, resid 1007\n", "\n", "-----------------------Summary of Estimated Coefficients------------------------\n", " Variable Coef Std Err t-stat p-value CI 2.5% CI 97.5%\n", "--------------------------------------------------------------------------------\n", " x 0.0338 0.0322 1.05 0.2944 -0.0293 0.0969\n", " intercept 0.0000 0.0000 0.50 0.6148 -0.0000 0.0000\n", "---------------------------------End of Summary---------------------------------\n" ] } ], "prompt_number": 32 }, { "cell_type": "code", "collapsed": false, "input": [ "mr = vwap / bars.open - 1\n", "mr = mr.between_time('9:30', '16:00')\n", "lagged = mr.tshift(1, 'min').between_time('9:30', '16:00')\n", "pd.ols(y=mr, x=lagged)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 33, "text": [ "\n", "-------------------------Summary of Regression Analysis-------------------------\n", "\n", "Formula: Y ~ + \n", "\n", "Number of Observations: 1007\n", "Number of Degrees of Freedom: 2\n", "\n", "R-squared: 0.0024\n", "Adj R-squared: 0.0014\n", "\n", "Rmse: 0.0001\n", "\n", "F-stat (1, 1005): 2.4262, p-value: 0.1196\n", "\n", "Degrees of Freedom: model 1, resid 1005\n", "\n", "-----------------------Summary of Estimated Coefficients------------------------\n", " Variable Coef Std Err t-stat p-value CI 2.5% CI 97.5%\n", "--------------------------------------------------------------------------------\n", " x 0.0503 0.0323 1.56 0.1196 -0.0130 0.1136\n", " intercept 0.0000 0.0000 0.66 0.5120 -0.0000 0.0000\n", "---------------------------------End of Summary---------------------------------\n" ] } ], "prompt_number": 33 }, { "cell_type": "code", "collapsed": false, "input": [ "inter = mr * vol\n", "inter = inter.between_time('9:30', '16:00')\n", "lagged_inter = inter.tshift(1, 'min').between_time('9:30', '16:00')\n", "pd.ols(y=mr, x=lagged_inter)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 34, "text": [ "\n", "-------------------------Summary of Regression Analysis-------------------------\n", "\n", "Formula: Y ~ + \n", "\n", "Number of Observations: 919\n", "Number of Degrees of Freedom: 2\n", "\n", "R-squared: 0.0055\n", "Adj R-squared: 0.0044\n", "\n", "Rmse: 0.0001\n", "\n", "F-stat (1, 917): 5.0453, p-value: 0.0249\n", "\n", "Degrees of Freedom: model 1, resid 917\n", "\n", "-----------------------Summary of Estimated Coefficients------------------------\n", " Variable Coef Std Err t-stat p-value CI 2.5% CI 97.5%\n", "--------------------------------------------------------------------------------\n", " x -0.0000 0.0000 -2.25 0.0249 -0.0000 -0.0000\n", " intercept 0.0000 0.0000 0.98 0.3251 -0.0000 0.0000\n", "---------------------------------End of Summary---------------------------------\n" ] } ], "prompt_number": 34 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Convert to percentage volume" ] }, { "cell_type": "code", "collapsed": false, "input": [ "vol = vol.groupby(vol.index.day).transform(lambda x: x/x.sum())\n", "vol.head()" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 35, "text": [ "Timestamp\n", "2011-11-01 10:00:00 0.000115\n", "2011-11-01 10:01:00 0.010772\n", "2011-11-01 10:02:00 0.038624\n", "2011-11-01 10:03:00 0.001148\n", "2011-11-01 10:04:00 0.001205" ] } ], "prompt_number": 35 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Verify" ] }, { "cell_type": "code", "collapsed": false, "input": [ "vol.resample('D', how='sum')" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 36, "text": [ "Timestamp\n", "2011-11-02 1\n", "2011-11-03 1\n", "2011-11-04 1\n", "Freq: D" ] } ], "prompt_number": 36 }, { "cell_type": "code", "collapsed": false, "input": [ "inter = mr * vol\n", "inter = inter.between_time('9:30', '16:00')\n", "lagged_inter = inter.tshift(1, 'min').between_time('9:30', '16:00')\n", "pd.ols(y=mr, x=lagged_inter)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 37, "text": [ "\n", "-------------------------Summary of Regression Analysis-------------------------\n", "\n", "Formula: Y ~ + \n", "\n", "Number of Observations: 919\n", "Number of Degrees of Freedom: 2\n", "\n", "R-squared: 0.0050\n", "Adj R-squared: 0.0039\n", "\n", "Rmse: 0.0001\n", "\n", "F-stat (1, 917): 4.6031, p-value: 0.0322\n", "\n", "Degrees of Freedom: model 1, resid 917\n", "\n", "-----------------------Summary of Estimated Coefficients------------------------\n", " Variable Coef Std Err t-stat p-value CI 2.5% CI 97.5%\n", "--------------------------------------------------------------------------------\n", " x -4.1357 1.9276 -2.15 0.0322 -7.9139 -0.3576\n", " intercept 0.0000 0.0000 0.98 0.3283 -0.0000 0.0000\n", "---------------------------------End of Summary---------------------------------\n" ] } ], "prompt_number": 37 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Vivaldi FTW" ] }, { "cell_type": "code", "collapsed": false, "input": [ "hour = vol.index.hour\n", "hourly_volume = vol.groupby(hour).mean()" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 38 }, { "cell_type": "code", "collapsed": true, "input": [ "hourly_volume.plot(kind='bar')" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 39, "text": [ "" ] }, { "output_type": "display_data", "svg": [ "\n", "\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n" ], "text": [ "" ] } ], "prompt_number": 39 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Expanding window of hourly means for volume" ] }, { "cell_type": "code", "collapsed": false, "input": [ "hourly = vol.resample('H')\n", "\n", "def calc_mean(hr):\n", " hr = time(hour=hr)\n", " data = hourly.at_time(hr)\n", " return pd.expanding_mean(data)\n", "\n", "df = pd.concat([calc_mean(hr) for hr in range(10, 16)])\n", "df = df.sort_index()\n", "df" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 40, "text": [ "Timestamp\n", "2011-11-01 10:00:00 0.000115\n", "2011-11-01 11:00:00 0.002510\n", "2011-11-01 12:00:00 0.001253\n", "2011-11-01 13:00:00 0.000628\n", "2011-11-01 14:00:00 0.003013\n", "2011-11-01 15:00:00 0.004419\n", "2011-11-02 10:00:00 0.000742\n", "2011-11-02 11:00:00 0.002986\n", "2011-11-02 12:00:00 0.002283\n", "2011-11-02 13:00:00 0.001088\n", "2011-11-02 14:00:00 0.002224\n", "2011-11-02 15:00:00 0.003979\n", "2011-11-03 10:00:00 0.000495\n", "2011-11-03 11:00:00 0.002537\n", "2011-11-03 12:00:00 0.002102\n", "2011-11-03 13:00:00 0.002193\n", "2011-11-03 14:00:00 0.003233\n", "2011-11-03 15:00:00 0.003228" ] } ], "prompt_number": 40 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Compute deviations from the hourly means" ] }, { "cell_type": "code", "collapsed": false, "input": [ "clean_vol = vol.between_time('10:00', '15:59')\n", "dev = clean_vol - df.reindex(clean_vol.index, method='pad') # be careful over day boundaries\n", "dev" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 41, "text": [ "Timestamp\n", "2011-11-01 10:00:00 0.000000\n", "2011-11-01 10:01:00 0.010658\n", "2011-11-01 10:02:00 0.038509\n", "2011-11-01 10:03:00 0.001033\n", "2011-11-01 10:04:00 0.001091\n", "2011-11-01 10:05:00 0.001205\n", "2011-11-01 10:06:00 0.006492\n", "2011-11-01 10:07:00 0.001909\n", "2011-11-01 10:08:00 0.000307\n", "2011-11-01 10:09:00 0.000520\n", "2011-11-01 10:10:00 0.000918\n", "2011-11-01 10:11:00 0.000895\n", "2011-11-01 10:12:00 0.001585\n", "2011-11-01 10:13:00 0.001205\n", "2011-11-01 10:14:00 0.003225\n", "...\n", "2011-11-03 15:45:00 -0.002881\n", "2011-11-03 15:46:00 -0.002411\n", "2011-11-03 15:47:00 -0.002708\n", "2011-11-03 15:48:00 -0.002881\n", "2011-11-03 15:49:00 -0.001083\n", "2011-11-03 15:50:00 -0.003228\n", "2011-11-03 15:51:00 -0.001469\n", "2011-11-03 15:52:00 -0.003054\n", "2011-11-03 15:53:00 -0.003054\n", "2011-11-03 15:54:00 -0.003054\n", "2011-11-03 15:55:00 -0.002303\n", "2011-11-03 15:56:00 0.012786\n", "2011-11-03 15:57:00 -0.003054\n", "2011-11-03 15:58:00 -0.002521\n", "2011-11-03 15:59:00 -0.000219\n", "Length: 1080" ] } ], "prompt_number": 41 }, { "cell_type": "code", "collapsed": false, "input": [ "inter = mr * dev\n", "inter = inter.between_time('10:00', '15:59')\n", "pd.ols(y=mr, x=inter.tshift(1, 'min'))" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 42, "text": [ "\n", "-------------------------Summary of Regression Analysis-------------------------\n", "\n", "Formula: Y ~ + \n", "\n", "Number of Observations: 919\n", "Number of Degrees of Freedom: 2\n", "\n", "R-squared: 0.0059\n", "Adj R-squared: 0.0048\n", "\n", "Rmse: 0.0001\n", "\n", "F-stat (1, 917): 5.3977, p-value: 0.0204\n", "\n", "Degrees of Freedom: model 1, resid 917\n", "\n", "-----------------------Summary of Estimated Coefficients------------------------\n", " Variable Coef Std Err t-stat p-value CI 2.5% CI 97.5%\n", "--------------------------------------------------------------------------------\n", " x -4.6532 2.0029 -2.32 0.0204 -8.5789 -0.7276\n", " intercept 0.0000 0.0000 0.97 0.3299 -0.0000 0.0000\n", "---------------------------------End of Summary---------------------------------\n" ] } ], "prompt_number": 42 }, { "cell_type": "markdown", "metadata": {}, "source": [ "###Date range creation" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`pd.date_range`" ] }, { "cell_type": "code", "collapsed": false, "input": [ "rng = pd.date_range('2005', '2012', freq='M')\n", "rng" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 43, "text": [ "\n", "[2005-10-31 00:00:00, ..., 2012-09-30 00:00:00]\n", "Length: 84, Freq: M, Timezone: None" ] } ], "prompt_number": 43 }, { "cell_type": "code", "collapsed": false, "input": [ "pd.date_range('2005', periods=7*12, freq='M')" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 44, "text": [ "\n", "[2005-10-31 00:00:00, ..., 2012-09-30 00:00:00]\n", "Length: 84, Freq: M, Timezone: None" ] } ], "prompt_number": 44 }, { "cell_type": "code", "collapsed": false, "input": [ "pd.date_range(end='2012', periods=7*12, freq='M')" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 45, "text": [ "\n", "[2005-10-31 00:00:00, ..., 2012-09-30 00:00:00]\n", "Length: 84, Freq: M, Timezone: None" ] } ], "prompt_number": 45 }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Frequency constants\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
NameDescription
DCalendar day
BBusiness day
MCalendar end of month
MSCalendar start of month
BMBusiness end of month
BMSBusiness start of month
W-{MON, TUE,...}Week ending on Monday, Tuesday, ...
Q-{JAN, FEB,...}Quarter end with year ending January, February...
QS-{JAN, FEB,...}Quarter start with year ending January, February...
BQ-{JAN, FEB,...}Business quarter end with year ending January, February...
BQS-{JAN, FEB,...}Business quarter start with year ending January, February...
A-{JAN, FEB, ...}Year end (December)
AS-{JAN, FEB, ...}Year start (December)
BA-{JAN, FEB, ...}Business year end (December)
BAS-{JAN, FEB, ...}Business year start (December)
HHour
TMinute
sSecond
L, msMillisecond
UMicrosecond
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Anchored offsets" ] }, { "cell_type": "code", "collapsed": false, "input": [ "pd.date_range('2005', periods=4, freq='Q')" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 46, "text": [ "\n", "[2005-12-31 00:00:00, ..., 2006-09-30 00:00:00]\n", "Length: 4, Freq: Q-DEC, Timezone: None" ] } ], "prompt_number": 46 }, { "cell_type": "code", "collapsed": false, "input": [ "pd.date_range('2005', periods=4, freq='Q-NOV')" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 47, "text": [ "\n", "[2005-11-30 00:00:00, ..., 2006-08-31 00:00:00]\n", "Length: 4, Freq: Q-NOV, Timezone: None" ] } ], "prompt_number": 47 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Week anchor indicates end of week" ] }, { "cell_type": "code", "collapsed": false, "input": [ "wkrng = pd.date_range('2012-10-25', periods=3, freq='W')\n", "wkrng" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 48, "text": [ "\n", "[2012-10-28 00:00:00, ..., 2012-11-11 00:00:00]\n", "Length: 3, Freq: W-SUN, Timezone: None" ] } ], "prompt_number": 48 }, { "cell_type": "code", "collapsed": false, "input": [ "wkrng[0].dayofweek" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 49, "text": [ "6" ] } ], "prompt_number": 49 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Year anchor indicates year ending month" ] }, { "cell_type": "code", "collapsed": false, "input": [ "pd.date_range('2005', periods=3, freq='A-JUN')" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 50, "text": [ "\n", "[2006-06-30 00:00:00, ..., 2008-06-30 00:00:00]\n", "Length: 3, Freq: A-JUN, Timezone: None" ] } ], "prompt_number": 50 }, { "cell_type": "markdown", "metadata": {}, "source": [ "`DatetimeIndex` is a subclass of Index" ] }, { "cell_type": "code", "collapsed": false, "input": [ "isinstance(rng, pd.Index)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 51, "text": [ "True" ] } ], "prompt_number": 51 }, { "cell_type": "code", "collapsed": false, "input": [ "rng[2:4]" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 52, "text": [ "\n", "[2005-12-31 00:00:00, 2006-01-31 00:00:00]\n", "Length: 2, Freq: M, Timezone: None" ] } ], "prompt_number": 52 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Use it for `Series/DataFrame` labelling" ] }, { "cell_type": "code", "collapsed": false, "input": [ "s = Series(randn(len(rng)), rng)\n", "s.head()" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 53, "text": [ "2005-10-31 1.026565\n", "2005-11-30 -0.306949\n", "2005-12-31 -1.838600\n", "2006-01-31 -1.325459\n", "2006-02-28 1.642384\n", "Freq: M" ] } ], "prompt_number": 53 }, { "cell_type": "code", "collapsed": false, "input": [ "df = DataFrame(randn(len(rng), 3), rng, ['X', 'Y', 'Z'])\n", "df.head()" ], "language": "python", "metadata": {}, "outputs": [ { "html": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
XYZ
2005-10-31-0.256664 0.255390-0.765260
2005-11-30 2.035113 1.694331 1.436986
2005-12-31 1.094142-1.098397 0.643988
2006-01-31 0.714353 0.429468 0.702067
2006-02-28 0.318068 0.656602 1.701144
\n", "
" ], "output_type": "pyout", "prompt_number": 54, "text": [ " X Y Z\n", "2005-10-31 -0.256664 0.255390 -0.765260\n", "2005-11-30 2.035113 1.694331 1.436986\n", "2005-12-31 1.094142 -1.098397 0.643988\n", "2006-01-31 0.714353 0.429468 0.702067\n", "2006-02-28 0.318068 0.656602 1.701144" ] } ], "prompt_number": 54 }, { "cell_type": "markdown", "metadata": {}, "source": [ "####Label indexing" ] }, { "cell_type": "code", "collapsed": false, "input": [ "s[datetime(2005, 1, 31) : datetime(2006, 12, 31)] #slice end inclusive" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 55, "text": [ "2005-10-31 1.026565\n", "2005-11-30 -0.306949\n", "2005-12-31 -1.838600\n", "2006-01-31 -1.325459\n", "2006-02-28 1.642384\n", "2006-03-31 -0.274467\n", "2006-04-30 -0.433675\n", "2006-05-31 -0.963449\n", "2006-06-30 1.112873\n", "2006-07-31 -0.089778\n", "2006-08-31 0.046948\n", "2006-09-30 0.035094\n", "2006-10-31 1.113169\n", "2006-11-30 -0.401821\n", "2006-12-31 0.723640\n", "Freq: M" ] } ], "prompt_number": 55 }, { "cell_type": "code", "collapsed": false, "input": [ "df['2005-1-31':'2006-12-31']" ], "language": "python", "metadata": {}, "outputs": [ { "html": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
XYZ
2005-10-31-0.256664 0.255390-0.765260
2005-11-30 2.035113 1.694331 1.436986
2005-12-31 1.094142-1.098397 0.643988
2006-01-31 0.714353 0.429468 0.702067
2006-02-28 0.318068 0.656602 1.701144
2006-03-31 0.192433 1.127046-2.496724
2006-04-30 1.417775 0.845626-0.713990
2006-05-31 0.057333 0.016464-0.311196
2006-06-30-0.535289 0.131742 0.459589
2006-07-31-0.787894-1.482390-0.341181
2006-08-31 0.131652 2.037986-0.547023
2006-09-30 0.883980 1.178719-1.960677
2006-10-31 0.096330-0.034388 0.944521
2006-11-30-0.559477 0.178275-0.637401
2006-12-31-0.730724 0.363436-1.048955
\n", "
" ], "output_type": "pyout", "prompt_number": 56, "text": [ " X Y Z\n", "2005-10-31 -0.256664 0.255390 -0.765260\n", "2005-11-30 2.035113 1.694331 1.436986\n", "2005-12-31 1.094142 -1.098397 0.643988\n", "2006-01-31 0.714353 0.429468 0.702067\n", "2006-02-28 0.318068 0.656602 1.701144\n", "2006-03-31 0.192433 1.127046 -2.496724\n", "2006-04-30 1.417775 0.845626 -0.713990\n", "2006-05-31 0.057333 0.016464 -0.311196\n", "2006-06-30 -0.535289 0.131742 0.459589\n", "2006-07-31 -0.787894 -1.482390 -0.341181\n", "2006-08-31 0.131652 2.037986 -0.547023\n", "2006-09-30 0.883980 1.178719 -1.960677\n", "2006-10-31 0.096330 -0.034388 0.944521\n", "2006-11-30 -0.559477 0.178275 -0.637401\n", "2006-12-31 -0.730724 0.363436 -1.048955" ] } ], "prompt_number": 56 }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Partial indexing**" ] }, { "cell_type": "code", "collapsed": false, "input": [ "s['2005':'2006']" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 57, "text": [ "2005-10-31 1.026565\n", "2005-11-30 -0.306949\n", "2005-12-31 -1.838600\n", "2006-01-31 -1.325459\n", "2006-02-28 1.642384\n", "2006-03-31 -0.274467\n", "2006-04-30 -0.433675\n", "2006-05-31 -0.963449\n", "2006-06-30 1.112873\n", "2006-07-31 -0.089778\n", "2006-08-31 0.046948\n", "2006-09-30 0.035094\n", "2006-10-31 1.113169\n", "2006-11-30 -0.401821\n", "2006-12-31 0.723640\n", "Freq: M" ] } ], "prompt_number": 57 }, { "cell_type": "markdown", "metadata": {}, "source": [ "positional indexing still works" ] }, { "cell_type": "code", "collapsed": false, "input": [ "df[:2] # slice end exclusive" ], "language": "python", "metadata": {}, "outputs": [ { "html": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
XYZ
2005-10-31-0.256664 0.255390-0.765260
2005-11-30 2.035113 1.694331 1.436986
\n", "
" ], "output_type": "pyout", "prompt_number": 58, "text": [ " X Y Z\n", "2005-10-31 -0.256664 0.255390 -0.765260\n", "2005-11-30 2.035113 1.694331 1.436986" ] } ], "prompt_number": 58 }, { "cell_type": "markdown", "metadata": {}, "source": [ "####Elements of DatetimeIndex" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Elements boxed as `Timestamp` (subclass of `datetime.datetime`)" ] }, { "cell_type": "code", "collapsed": false, "input": [ "elm = rng[0]\n", "elm" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 59, "text": [ "" ] } ], "prompt_number": 59 }, { "cell_type": "code", "collapsed": false, "input": [ "isinstance(elm, datetime)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 60, "text": [ "True" ] } ], "prompt_number": 60 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Why do we need this subclass?" ] }, { "cell_type": "code", "collapsed": false, "input": [ "elm.nanosecond" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 61, "text": [ "0" ] } ], "prompt_number": 61 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Implemented internally using numpy.datetime64 (dtype='M8[ns]')" ] }, { "cell_type": "code", "collapsed": false, "input": [ "val = rng.values" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 62 }, { "cell_type": "code", "collapsed": false, "input": [ "type(val)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 63, "text": [ "numpy.ndarray" ] } ], "prompt_number": 63 }, { "cell_type": "code", "collapsed": false, "input": [ "val.dtype" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 64, "text": [ "dtype('datetime64[ns]')" ] } ], "prompt_number": 64 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Upgrade Numpy to 1.7b to fix repr issue" ] }, { "cell_type": "code", "collapsed": false, "input": [ "val[0]" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 65, "text": [ "1970-01-14 40:00:00" ] } ], "prompt_number": 65 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Or use DatetimeIndex.asobject for workaround" ] }, { "cell_type": "code", "collapsed": false, "input": [ "rng.asobject.values[0]" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 66, "text": [ "" ] } ], "prompt_number": 66 }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Other views" ] }, { "cell_type": "code", "collapsed": false, "input": [ "rng.asobject" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 67, "text": [ "Index([2005-10-31 00:00:00, 2005-11-30 00:00:00, 2005-12-31 00:00:00, 2006-01-31 00:00:00, 2006-02-28 00:00:00, 2006-03-31 00:00:00, 2006-04-30 00:00:00, 2006-05-31 00:00:00, 2006-06-30 00:00:00, 2006-07-31 00:00:00, 2006-08-31 00:00:00, 2006-09-30 00:00:00, 2006-10-31 00:00:00, 2006-11-30 00:00:00, 2006-12-31 00:00:00, 2007-01-31 00:00:00, 2007-02-28 00:00:00, 2007-03-31 00:00:00, 2007-04-30 00:00:00, 2007-05-31 00:00:00, 2007-06-30 00:00:00, 2007-07-31 00:00:00, 2007-08-31 00:00:00, 2007-09-30 00:00:00, 2007-10-31 00:00:00, 2007-11-30 00:00:00, 2007-12-31 00:00:00, 2008-01-31 00:00:00, 2008-02-29 00:00:00, 2008-03-31 00:00:00, 2008-04-30 00:00:00, 2008-05-31 00:00:00, 2008-06-30 00:00:00, 2008-07-31 00:00:00, 2008-08-31 00:00:00, 2008-09-30 00:00:00, 2008-10-31 00:00:00, 2008-11-30 00:00:00, 2008-12-31 00:00:00, 2009-01-31 00:00:00, 2009-02-28 00:00:00, 2009-03-31 00:00:00, 2009-04-30 00:00:00, 2009-05-31 00:00:00, 2009-06-30 00:00:00, 2009-07-31 00:00:00, 2009-08-31 00:00:00, 2009-09-30 00:00:00, 2009-10-31 00:00:00, 2009-11-30 00:00:00, 2009-12-31 00:00:00, 2010-01-31 00:00:00, 2010-02-28 00:00:00, 2010-03-31 00:00:00, 2010-04-30 00:00:00, 2010-05-31 00:00:00, 2010-06-30 00:00:00, 2010-07-31 00:00:00, 2010-08-31 00:00:00, 2010-09-30 00:00:00, 2010-10-31 00:00:00, 2010-11-30 00:00:00, 2010-12-31 00:00:00, 2011-01-31 00:00:00, 2011-02-28 00:00:00, 2011-03-31 00:00:00, 2011-04-30 00:00:00, 2011-05-31 00:00:00, 2011-06-30 00:00:00, 2011-07-31 00:00:00, 2011-08-31 00:00:00, 2011-09-30 00:00:00, 2011-10-31 00:00:00, 2011-11-30 00:00:00, 2011-12-31 00:00:00, 2012-01-31 00:00:00, 2012-02-29 00:00:00, 2012-03-31 00:00:00, 2012-04-30 00:00:00, 2012-05-31 00:00:00, 2012-06-30 00:00:00, 2012-07-31 00:00:00, 2012-08-31 00:00:00, 2012-09-30 00:00:00], dtype=object)" ] } ], "prompt_number": 67 }, { "cell_type": "code", "collapsed": false, "input": [ "rng.to_pydatetime()" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 68, "text": [ "array([2005-10-31 00:00:00, 2005-11-30 00:00:00, 2005-12-31 00:00:00,\n", " 2006-01-31 00:00:00, 2006-02-28 00:00:00, 2006-03-31 00:00:00,\n", " 2006-04-30 00:00:00, 2006-05-31 00:00:00, 2006-06-30 00:00:00,\n", " 2006-07-31 00:00:00, 2006-08-31 00:00:00, 2006-09-30 00:00:00,\n", " 2006-10-31 00:00:00, 2006-11-30 00:00:00, 2006-12-31 00:00:00,\n", " 2007-01-31 00:00:00, 2007-02-28 00:00:00, 2007-03-31 00:00:00,\n", " 2007-04-30 00:00:00, 2007-05-31 00:00:00, 2007-06-30 00:00:00,\n", " 2007-07-31 00:00:00, 2007-08-31 00:00:00, 2007-09-30 00:00:00,\n", " 2007-10-31 00:00:00, 2007-11-30 00:00:00, 2007-12-31 00:00:00,\n", " 2008-01-31 00:00:00, 2008-02-29 00:00:00, 2008-03-31 00:00:00,\n", " 2008-04-30 00:00:00, 2008-05-31 00:00:00, 2008-06-30 00:00:00,\n", " 2008-07-31 00:00:00, 2008-08-31 00:00:00, 2008-09-30 00:00:00,\n", " 2008-10-31 00:00:00, 2008-11-30 00:00:00, 2008-12-31 00:00:00,\n", " 2009-01-31 00:00:00, 2009-02-28 00:00:00, 2009-03-31 00:00:00,\n", " 2009-04-30 00:00:00, 2009-05-31 00:00:00, 2009-06-30 00:00:00,\n", " 2009-07-31 00:00:00, 2009-08-31 00:00:00, 2009-09-30 00:00:00,\n", " 2009-10-31 00:00:00, 2009-11-30 00:00:00, 2009-12-31 00:00:00,\n", " 2010-01-31 00:00:00, 2010-02-28 00:00:00, 2010-03-31 00:00:00,\n", " 2010-04-30 00:00:00, 2010-05-31 00:00:00, 2010-06-30 00:00:00,\n", " 2010-07-31 00:00:00, 2010-08-31 00:00:00, 2010-09-30 00:00:00,\n", " 2010-10-31 00:00:00, 2010-11-30 00:00:00, 2010-12-31 00:00:00,\n", " 2011-01-31 00:00:00, 2011-02-28 00:00:00, 2011-03-31 00:00:00,\n", " 2011-04-30 00:00:00, 2011-05-31 00:00:00, 2011-06-30 00:00:00,\n", " 2011-07-31 00:00:00, 2011-08-31 00:00:00, 2011-09-30 00:00:00,\n", " 2011-10-31 00:00:00, 2011-11-30 00:00:00, 2011-12-31 00:00:00,\n", " 2012-01-31 00:00:00, 2012-02-29 00:00:00, 2012-03-31 00:00:00,\n", " 2012-04-30 00:00:00, 2012-05-31 00:00:00, 2012-06-30 00:00:00,\n", " 2012-07-31 00:00:00, 2012-08-31 00:00:00, 2012-09-30 00:00:00], dtype=object)" ] } ], "prompt_number": 68 }, { "cell_type": "code", "collapsed": false, "input": [ "rng.to_pydatetime()[0]" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 69, "text": [ "datetime.datetime(2005, 10, 31, 0, 0)" ] } ], "prompt_number": 69 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Integer representation" ] }, { "cell_type": "code", "collapsed": false, "input": [ "type(rng.asi8)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 70, "text": [ "numpy.ndarray" ] } ], "prompt_number": 70 }, { "cell_type": "code", "collapsed": false, "input": [ "rng.asi8.dtype" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 71, "text": [ "dtype('int64')" ] } ], "prompt_number": 71 }, { "cell_type": "code", "collapsed": false, "input": [ "rng.asi8[0]" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 72, "text": [ "1130716800000000000" ] } ], "prompt_number": 72 }, { "cell_type": "markdown", "metadata": {}, "source": [ "###More fun with resampling and asfreq" ] }, { "cell_type": "code", "collapsed": false, "input": [ "s.index.freqstr" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 73, "text": [ "'M'" ] } ], "prompt_number": 73 }, { "cell_type": "code", "collapsed": false, "input": [ "s.resample('30D').head(10)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 74, "text": [ "2005-10-31 1.026565\n", "2005-11-30 -0.306949\n", "2005-12-30 NaN\n", "2006-01-29 NaN\n", "2006-02-28 1.642384\n", "2006-03-30 NaN\n", "2006-04-29 NaN\n", "2006-05-29 NaN\n", "2006-06-28 NaN\n", "2006-07-28 NaN\n", "Freq: 30D" ] } ], "prompt_number": 74 }, { "cell_type": "code", "collapsed": false, "input": [ "s.resample('30D', fill_method='ffill').head(10)" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 75, "text": [ "2005-10-31 1.026565\n", "2005-11-30 -0.306949\n", "2005-12-30 -0.306949\n", "2006-01-29 -1.838600\n", "2006-02-28 1.642384\n", "2006-03-30 1.642384\n", "2006-04-29 -0.274467\n", "2006-05-29 -0.433675\n", "2006-06-28 -0.963449\n", "2006-07-28 1.112873\n", "Freq: 30D" ] } ], "prompt_number": 75 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Upsampling" ] }, { "cell_type": "code", "collapsed": false, "input": [ "s.ix[:3].resample('W')" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 76, "text": [ "2005-11-06 NaN\n", "2005-11-13 NaN\n", "2005-11-20 NaN\n", "2005-11-27 NaN\n", "2005-12-04 NaN\n", "2005-12-11 NaN\n", "2005-12-18 NaN\n", "2005-12-25 NaN\n", "2006-01-01 NaN\n", "Freq: W-SUN" ] } ], "prompt_number": 76 }, { "cell_type": "code", "collapsed": false, "input": [ "s.ix[:3].resample('W', fill_method='ffill')" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 77, "text": [ "2005-11-06 1.026565\n", "2005-11-13 1.026565\n", "2005-11-20 1.026565\n", "2005-11-27 1.026565\n", "2005-12-04 -0.306949\n", "2005-12-11 -0.306949\n", "2005-12-18 -0.306949\n", "2005-12-25 -0.306949\n", "2006-01-01 -1.838600\n", "Freq: W-SUN" ] } ], "prompt_number": 77 }, { "cell_type": "markdown", "metadata": {}, "source": [ "asfreq" ] }, { "cell_type": "code", "collapsed": false, "input": [ "s.asfreq('Q').head()" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 78, "text": [ "2005-12-31 -1.838600\n", "2006-03-31 -0.274467\n", "2006-06-30 1.112873\n", "2006-09-30 0.035094\n", "2006-12-31 0.723640\n", "Freq: Q-DEC" ] } ], "prompt_number": 78 }, { "cell_type": "code", "collapsed": false, "input": [ "s.resample('Q', 'last').head()" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 79, "text": [ "2005-12-31 -1.838600\n", "2006-03-31 -0.274467\n", "2006-06-30 1.112873\n", "2006-09-30 0.035094\n", "2006-12-31 0.723640\n", "Freq: Q-DEC" ] } ], "prompt_number": 79 }, { "cell_type": "markdown", "metadata": {}, "source": [ "closed: 'left' or 'right' bin edge is closed (default is 'right')" ] }, { "cell_type": "code", "collapsed": false, "input": [ "s.resample('Q').head()" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 80, "text": [ "2005-12-31 -0.372995\n", "2006-03-31 0.014153\n", "2006-06-30 -0.094750\n", "2006-09-30 -0.002579\n", "2006-12-31 0.478330\n", "Freq: Q-DEC" ] } ], "prompt_number": 80 }, { "cell_type": "code", "collapsed": false, "input": [ "s.ix[3:6].mean()" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 81, "text": [ "0.014152740332844638" ] } ], "prompt_number": 81 }, { "cell_type": "code", "collapsed": false, "input": [ "s.resample('Q', closed='left').head()" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 82, "text": [ "2005-12-31 0.359808\n", "2006-03-31 -0.507225\n", "2006-06-30 -0.557197\n", "2006-09-30 0.356681\n", "2006-12-31 0.248814\n", "Freq: Q-DEC" ] } ], "prompt_number": 82 }, { "cell_type": "code", "collapsed": false, "input": [ "s.ix[2:5].mean()" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 83, "text": [ "-0.50722477768944352" ] } ], "prompt_number": 83 }, { "cell_type": "markdown", "metadata": {}, "source": [ "label: label the bin with 'left' or 'right' edge (default is 'right')" ] }, { "cell_type": "code", "collapsed": false, "input": [ "s.resample('Q').head()" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 84, "text": [ "2005-12-31 -0.372995\n", "2006-03-31 0.014153\n", "2006-06-30 -0.094750\n", "2006-09-30 -0.002579\n", "2006-12-31 0.478330\n", "Freq: Q-DEC" ] } ], "prompt_number": 84 }, { "cell_type": "code", "collapsed": false, "input": [ "s.resample('Q', label='left').head()" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 85, "text": [ "2005-09-30 -0.372995\n", "2005-12-31 0.014153\n", "2006-03-31 -0.094750\n", "2006-06-30 -0.002579\n", "2006-09-30 0.478330\n", "Freq: Q-DEC" ] } ], "prompt_number": 85 }, { "cell_type": "markdown", "metadata": {}, "source": [ "loffset: shift the result index" ] }, { "cell_type": "code", "collapsed": false, "input": [ "s.resample('Q', label='left', loffset='-1D').head()" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 86, "text": [ "2005-09-29 -0.372995\n", "2005-12-30 0.014153\n", "2006-03-30 -0.094750\n", "2006-06-29 -0.002579\n", "2006-09-29 0.478330" ] } ], "prompt_number": 86 }, { "cell_type": "markdown", "metadata": {}, "source": [ "###Time zones" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "####Localization" ] }, { "cell_type": "code", "collapsed": false, "input": [ "rng.tz" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 87 }, { "cell_type": "code", "collapsed": false, "input": [ "d = rng[0]\n", "d" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 88, "text": [ "" ] } ], "prompt_number": 88 }, { "cell_type": "code", "collapsed": false, "input": [ "d.tz" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 89 }, { "cell_type": "code", "collapsed": false, "input": [ "localized = rng.tz_localize('US/Eastern')" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 90 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Localization assumes naive time is local (and not UTC)" ] }, { "cell_type": "code", "collapsed": false, "input": [ "localized[0]" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 91, "text": [ "" ] } ], "prompt_number": 91 }, { "cell_type": "code", "collapsed": false, "input": [ "localized.asi8[0]" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 92, "text": [ "1130734800000000000" ] } ], "prompt_number": 92 }, { "cell_type": "code", "collapsed": false, "input": [ "rng.asi8[0]" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 93, "text": [ "1130716800000000000" ] } ], "prompt_number": 93 }, { "cell_type": "code", "collapsed": false, "input": [ "d_utc = d.tz_localize('UTC')\n", "d_utc" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 94, "text": [ "" ] } ], "prompt_number": 94 }, { "cell_type": "code", "collapsed": false, "input": [ "d_utc.tz_localize('US/Eastern')" ], "language": "python", "metadata": {}, "outputs": [ { "ename": "Exception", "evalue": "Cannot localize tz-aware Timestamp, use tz_convert for conversions", "output_type": "pyerr", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m\n\u001b[0;31mException\u001b[0m Traceback (most recent call last)", "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0md_utc\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtz_localize\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'US/Eastern'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m", "\u001b[0;32m/Users/changshe/code/pandas/pandas/lib.so\u001b[0m in \u001b[0;36mpandas.lib.Timestamp.tz_localize (pandas/src/tseries.c:32388)\u001b[0;34m()\u001b[0m\n", "\u001b[0;31mException\u001b[0m: Cannot localize tz-aware Timestamp, use tz_convert for conversions" ] } ], "prompt_number": 95 }, { "cell_type": "markdown", "metadata": {}, "source": [ "####TZ conversions" ] }, { "cell_type": "code", "collapsed": false, "input": [ "localized.tz_convert('UTC')" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 96, "text": [ "\n", "[2005-10-31 05:00:00, ..., 2012-09-30 04:00:00]\n", "Length: 84, Freq: M, Timezone: UTC" ] } ], "prompt_number": 96 }, { "cell_type": "code", "collapsed": false, "input": [ "d_ny = d_utc.tz_convert('US/Eastern')\n", "d_ny" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 97, "text": [ "" ] } ], "prompt_number": 97 }, { "cell_type": "code", "collapsed": false, "input": [ "rng.tz_convert('US/Eastern')" ], "language": "python", "metadata": {}, "outputs": [ { "ename": "Exception", "evalue": "Cannot convert tz-naive timestamps, use tz_localize to localize", "output_type": "pyerr", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m\n\u001b[0;31mException\u001b[0m Traceback (most recent call last)", "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mrng\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtz_convert\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'US/Eastern'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m", "\u001b[0;32m/Users/changshe/code/pandas/pandas/tseries/index.pyc\u001b[0m in \u001b[0;36mtz_convert\u001b[0;34m(self, tz)\u001b[0m\n\u001b[1;32m 1275\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtz\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1276\u001b[0m \u001b[0;31m# tz naive, use tz_localize\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1277\u001b[0;31m raise Exception('Cannot convert tz-naive timestamps, use '\n\u001b[0m\u001b[1;32m 1278\u001b[0m 'tz_localize to localize')\n\u001b[1;32m 1279\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;31mException\u001b[0m: Cannot convert tz-naive timestamps, use tz_localize to localize" ] } ], "prompt_number": 98 }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Period representation" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A lot of time series data is better represented as intervals of time rather than points in time.\n", "\n", "This is represented in _pandas_ as Period and PeriodIndex" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "####Creating periods" ] }, { "cell_type": "code", "collapsed": false, "input": [ "p = pd.Period('2005', 'A')\n", "p" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 99, "text": [ "Period('2005', 'A-DEC')" ] } ], "prompt_number": 99 }, { "cell_type": "code", "collapsed": false, "input": [ "pd.Period('2006Q1', 'Q-MAR')" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 100, "text": [ "Period('2006Q1', 'Q-MAR')" ] } ], "prompt_number": 100 }, { "cell_type": "code", "collapsed": false, "input": [ "pd.Period('2007-1-1', 'B')" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 101, "text": [ "Period('2007-01-01', 'B')" ] } ], "prompt_number": 101 }, { "cell_type": "markdown", "metadata": {}, "source": [ "No xxx-start frequencies" ] }, { "cell_type": "code", "collapsed": false, "input": [ "pd.Period('2005', 'AS')" ], "language": "python", "metadata": {}, "outputs": [ { "ename": "KeyError", "evalue": "'AS-JAN'", "output_type": "pyerr", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m\n\u001b[0;31mKeyError\u001b[0m Traceback (most recent call last)", "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mpd\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mPeriod\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'2005'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'AS'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m", "\u001b[0;32m/Users/changshe/code/pandas/pandas/tseries/period.pyc\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, value, freq, ordinal, year, month, quarter, day, hour, minute, second)\u001b[0m\n\u001b[1;32m 114\u001b[0m \u001b[0;32mraise\u001b[0m \u001b[0mValueError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmsg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 115\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 116\u001b[0;31m \u001b[0mbase\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_gfc\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfreq\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 117\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mmult\u001b[0m \u001b[0;34m!=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 118\u001b[0m \u001b[0;32mraise\u001b[0m \u001b[0mValueError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Only mult == 1 supported'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m/Users/changshe/code/pandas/pandas/tseries/frequencies.pyc\u001b[0m in \u001b[0;36mget_freq_code\u001b[0;34m(freqstr)\u001b[0m\n\u001b[1;32m 71\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 72\u001b[0m \u001b[0mbase\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstride\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_base_and_stride\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfreqstr\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 73\u001b[0;31m \u001b[0mcode\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_period_str_to_code\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mbase\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 74\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 75\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mcode\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstride\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m/Users/changshe/code/pandas/pandas/tseries/frequencies.pyc\u001b[0m in \u001b[0;36m_period_str_to_code\u001b[0;34m(freqstr)\u001b[0m\n\u001b[1;32m 694\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0m_period_code_map\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mfreqstr\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 695\u001b[0m \u001b[0;32mexcept\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 696\u001b[0;31m \u001b[0malias\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_period_alias_dict\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mfreqstr\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 697\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0m_period_code_map\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0malias\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 698\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;31mKeyError\u001b[0m: 'AS-JAN'" ] } ], "prompt_number": 102 }, { "cell_type": "markdown", "metadata": {}, "source": [ "####PeriodRange" ] }, { "cell_type": "code", "collapsed": false, "input": [ "pd.period_range('2005', '2012', freq='A')" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 103, "text": [ "\n", "freq: A-DEC\n", "[2005, ..., 2012]\n", "length: 8" ] } ], "prompt_number": 103 }, { "cell_type": "code", "collapsed": false, "input": [ "prng = pd.period_range('2005', periods=7, freq='A')\n", "prng" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 104, "text": [ "\n", "freq: A-DEC\n", "[2005, ..., 2011]\n", "length: 7" ] } ], "prompt_number": 104 }, { "cell_type": "markdown", "metadata": {}, "source": [ "####Converting between representations" ] }, { "cell_type": "code", "collapsed": false, "input": [ "p" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 105, "text": [ "Period('2005', 'A-DEC')" ] } ], "prompt_number": 105 }, { "cell_type": "code", "collapsed": false, "input": [ "p.to_timestamp()" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 106, "text": [ "" ] } ], "prompt_number": 106 }, { "cell_type": "code", "collapsed": false, "input": [ "p.to_timestamp('M', 's')" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 107, "text": [ "" ] } ], "prompt_number": 107 }, { "cell_type": "code", "collapsed": false, "input": [ "p.to_timestamp('M', 'e')" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 108, "text": [ "" ] } ], "prompt_number": 108 }, { "cell_type": "code", "collapsed": false, "input": [ "prng.to_timestamp(how='e')" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 109, "text": [ "\n", "[2005-12-31 00:00:00, ..., 2011-12-31 00:00:00]\n", "Length: 7, Freq: A-DEC, Timezone: None" ] } ], "prompt_number": 109 }, { "cell_type": "code", "collapsed": false, "input": [ "prng.to_timestamp('M', 'e')" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 110, "text": [ "\n", "[2005-12-31 00:00:00, ..., 2011-12-31 00:00:00]\n", "Length: 7, Freq: A-DEC, Timezone: None" ] } ], "prompt_number": 110 }, { "cell_type": "code", "collapsed": false, "input": [ "rng" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 111, "text": [ "\n", "[2005-10-31 00:00:00, ..., 2012-09-30 00:00:00]\n", "Length: 84, Freq: M, Timezone: None" ] } ], "prompt_number": 111 }, { "cell_type": "code", "collapsed": false, "input": [ "rng.to_period()" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 112, "text": [ "\n", "freq: M\n", "[2005-10, ..., 2012-09]\n", "length: 84" ] } ], "prompt_number": 112 }, { "cell_type": "code", "collapsed": false, "input": [ "rng.to_period('D')" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 113, "text": [ "\n", "freq: D\n", "[2005-10-31, ..., 2012-09-30]\n", "length: 84" ] } ], "prompt_number": 113 }, { "cell_type": "markdown", "metadata": {}, "source": [ "####Bugggg" ] }, { "cell_type": "code", "collapsed": false, "input": [ "p" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 114, "text": [ "Period('2005', 'A-DEC')" ] } ], "prompt_number": 114 }, { "cell_type": "code", "collapsed": false, "input": [ "p.end_time" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 115, "text": [ "" ] } ], "prompt_number": 115 }, { "cell_type": "code", "collapsed": false, "input": [ "datetime(2005, 12, 31, 10, 0, 0) < p.end_time # WAT?!" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "pyout", "prompt_number": 116, "text": [ "False" ] } ], "prompt_number": 116 }, { "cell_type": "markdown", "metadata": {}, "source": [ "Look for a 0.9.1 bugfix release next week" ] }, { "cell_type": "code", "collapsed": false, "input": [], "language": "python", "metadata": {}, "outputs": [] } ], "metadata": {} } ] }