博客
关于我
理解Python系统下的时间格式
阅读量:350 次
发布时间:2019-03-04

本文共 3102 字,大约阅读时间需要 10 分钟。

  • Overview

    pandas/numpy/datetime/time,这四个module是常用的时间相关模块。timestampdatetimestr是三大类常用的数据类型。需要理顺彼此之间错综复杂的关系。

    The Python world has a number of avaiable representations of dates, times, deltas, and timespans.

  • Native Python dates and times: datetime and dateutil

    Python’s basic objects for working with dates and times reside in the built-in datetime module.

    Third-party dateutil can be used to parse dates from a variety of string formats.

    • The datetime module supplies classes for manipulating dates and times.

    • The dateutil module provides powerful extensions to the standard datetime module.

  • Typed arrays of times: Numpy's datetime64

    The weaknesses of Python’s datetime format inspired the Numpy team to add a set of native time series date type to Numpy.

    The datetime64 dtype encodes dates as 64-bit integers, and thus allows arrays of dates to be represented very compactly.

    The datetime64 requires a very specific input format.

    Because of the uniform type in NumPy datetime64 arrays, this type of operation can be accomplished much more quickly than if we were working directly with Python’s datetime objects.

    • Starting in NumPy 1.7, there are core array date types which natively support datetime functionality. The data type is called “datetime64”, so named because “datetime” is already taken by datetime library included in Python.

      The most basic way to create datetimes is from strings in ISO8601 date or datetime format.

      The Unit for internal storage is :

      1. automatically selected from the form of the string,
      2. and can be either :
        1. a unit: Y M W D
        2. a time unit: h m s ms us ns ps fs as

      datetime64 is the data type; datetime64[ns] or datetime64[s] or datetime64[unit] is datetime64 with unit.

      Finally, we will note that while the datetime64 data type addresses some of the deficiencies of the built-in Python datetime type, it lacks many of the convenient methods and functions provided by datetime and especially dateutil.

  • Dates and times in pandas: best of both worlds

    Pandas builds upon all the tools just discussed to provide Timestamp object, which combines the ease-of-use of datetime and dateutil with the efficient storage and vectorized interface of numpy.datetime64.

    From a group of these Timestamp objects, Pandas can construct a DatetimeIndex that can be used to index data in a Series or DataFrame.

    Pandas Time Series: Indexing by Time

    Where the Pandas time series tools become useful is when you begin to index data by timestamps.

    Pandas Time Series Data Structures

    For timestamps, Pandas provides the Timestamp type: it is essentially a replacement for Python’s native datetime, but is based on the more efficient numpy.datetime64 date type.

    For time Periods, Pandas provides the Period type, based on numpy.datetime64.

    For time deltas or durations, Pandas provides the Timedelta type, based on numpy.timedelta64, more efficient replacement for Python’s native datetime.timedelta type.

  • 汇总

    Python native is datetime.datetime data type from module: datetime;

    更高效的是datetime64 data type from module: NumPy;

    结合上述两者优点的是TimeStamp / Timedelta data type from module: Pandas;

  • 不同数据类型之间的转换

    在这里插入图片描述

  • References

转载地址:http://vtge.baihongyu.com/

你可能感兴趣的文章
Nifi同步过程中报错create_time字段找不到_实际目标表和源表中没有这个字段---大数据之Nifi工作笔记0066
查看>>
NIFI大数据进阶_FlowFile拓扑_对FlowFile内容和属性的修改删除添加_介绍和描述_以及实际操作---大数据之Nifi工作笔记0023
查看>>
NIFI大数据进阶_FlowFile生成器_GenerateFlowFile处理器_ReplaceText处理器_处理器介绍_处理过程说明---大数据之Nifi工作笔记0019
查看>>
NIFI大数据进阶_Json内容转换为Hive支持的文本格式_操作方法说明_01_EvaluteJsonPath处理器---大数据之Nifi工作笔记0031
查看>>
NIFI大数据进阶_Kafka使用相关说明_实际操作Kafka消费者处理器_来消费kafka数据---大数据之Nifi工作笔记0037
查看>>
NIFI大数据进阶_Kafka使用相关说明_实际操作Kafka生产者---大数据之Nifi工作笔记0036
查看>>
NIFI大数据进阶_NIFI的模板和组的使用-介绍和实际操作_创建组_嵌套组_模板创建下载_导入---大数据之Nifi工作笔记0022
查看>>
NIFI大数据进阶_NIFI监控功能实际操作_Summary查看系统和处理器运行情况_viewDataProvenance查看_---大数据之Nifi工作笔记0026
查看>>
NIFI大数据进阶_NIFI监控的强大功能介绍_处理器面板_进程组面板_summary监控_data_provenance事件源---大数据之Nifi工作笔记0025
查看>>
NIFI大数据进阶_NIFI集群知识点_认识NIFI集群以及集群的组成部分---大数据之Nifi工作笔记0014
查看>>
NIFI大数据进阶_NIFI集群知识点_集群的断开_重连_退役_卸载_总结---大数据之Nifi工作笔记0018
查看>>
NIFI大数据进阶_内嵌ZK模式集群1_搭建过程说明---大数据之Nifi工作笔记0015
查看>>
NIFI大数据进阶_外部ZK模式集群1_实际操作搭建NIFI外部ZK模式集群---大数据之Nifi工作笔记0017
查看>>
NIFI大数据进阶_实时同步MySql的数据到Hive中去_可增量同步_实时监控MySql数据库变化_操作方法说明_01---大数据之Nifi工作笔记0033
查看>>
NIFI大数据进阶_离线同步MySql数据到HDFS_01_实际操作---大数据之Nifi工作笔记0029
查看>>
NIFI大数据进阶_离线同步MySql数据到HDFS_02_实际操作_splitjson处理器_puthdfs处理器_querydatabasetable处理器---大数据之Nifi工作笔记0030
查看>>
NIFI大数据进阶_离线同步MySql数据到HDFS_说明操作步骤---大数据之Nifi工作笔记0028
查看>>
NIFI大数据进阶_连接与关系_设置数据流负载均衡_设置背压_设置展现弯曲_介绍以及实际操作---大数据之Nifi工作笔记0027
查看>>
NIFI数据库同步_多表_特定表同时同步_实际操作_MySqlToMysql_可推广到其他数据库_Postgresql_Hbase_SqlServer等----大数据之Nifi工作笔记0053
查看>>
NIFI汉化_替换logo_二次开发_Idea编译NIFI最新源码_详细过程记录_全解析_Maven编译NIFI避坑指南001---大数据之Nifi工作笔记0068
查看>>