So I looked into datetime64 a bit more and found this "NEP" for it:
https://github.com/numpy/numpy/blob/master/doc/neps/datetime-proposal.rst
The proposal purports to add two types datetime64 and timedelta64 but
actually adds 28: a datetime64 and timedelta64 types for each supported
time unit, from years down to attoseconds. Moreover, there's no good way to
implement code for these generically, because there's a fixed set of units
and the ones greater than seconds don't always consist of a fixed number of
seconds (e.g. there's a variable number of seconds in a year). I was hoping
for something significantly less complicated.
A slightly different (partially compatible) design would allow a single
parametric type to be at the core of all time operations. Something like
this:
bitstype 64 TimeDelta{p}
This is a bit like a NumPy timedelta64 except that the the units are
seconds*10^-p. That way, you can generically write all the code for
TimeDeltas instead of having to write code for each unit separately. It's
also more flexible since you don't need to write any new code to get
precisions like zeptoseconds or yoctoseconds. Since 2^63 seconds is 292
billion years and the universe has only been around for 14 billion or so
years, it seems unlikely that there will be much call for computations with
p < 0, although we can, of course, write the code to can handle that too.
As a bonus, unix timestamps have the identical in-memory representation as
TimeDelta{0} values. For units that are smaller than seconds, NumPy
datetime64 values have identical in-memory representation as corresponding
TimeDelta types.
[Aside: I would argue that letting "1 year" be a time delta is
fundamentally broken because "1 year" is not any fixed length of time. If
you want to add or subtract years, you are really doing integer arithmetic
and then interpreting the result as a number of years, not actually adding
duration of time.]
Since an absolute time is really just a time delta relative to a fixed
time, you can pretty easily define Time{p} in terms of TimeDelta{p} and
inherit all the functionality, just changing the presentation (how they
print, etc.) and a few other behaviors. In essence, all time values are
deltas, but some of those deltas are "free" and some are "fixed" relative
to a specific point in time, namely the UNIX epoch. The fixed ones know how
to display themselves as human-readable times whereas the free ones display
themselves as durations.
Operations with TimeDeltas are dirt simple: they just work like integers.
Operations with mixed p values have to be a little more tricky, but not too
badly so. NumPy opted for a "keep the precision" rule when using mixed
values of p and that seems sane so we might as well do the same thing. The
hard parts of all of this is the presentation: "what are the year, month,
and day, hour, minute, and second portions of that number of attoseconds?"
If we can write the code to compute those things, then I think we're
basically good. Oh yeah, and time zones. Those bastards.
Post by MilktraderOkay, I'll look into it. Pandas (Wes) and xtime (Jeff Ryan) should be good
places to look also.
Post by Stefan KarpinskiI think the NumPy datetime64 thing is pretty well thought through, not
that complicated, and good for high-performance numerical stuff which the
"time components" style representations are not good for (not compact,
operations are very slow). Since the design work is already done, once
someone groks that, implementing it shouldn't be all that hard in Julia as
the language lets you add bits types so easily.
Let's at least explore the NumPy-compatible option first. Dan, would you
be willing to do some research on the NumPy datetime64 type and figure out
how it works? My impression is that it represents timestamps as integers at
some unit of granularity. I'm unclear on whether it has flexible units of
granularity or if its fixed. I haven't found a real spec yet either (after
a very cursory bit of googling). We should maybe drop Wes McKinney a line
since I'm pretty sure he knows all about this.
Post by John Myles WhiteI feel like we should just implement something simple (maybe using
Avik's work) and get people to push on it until it breaks, then propose a
clean, finalized design after we see where pain points exist.
-- John
I've thought about this a bit and these are the time-related bits types
1. A 64-bit timestamp compatible with NumPy's still-experimental
datetime64 type<http://docs.scipy.org/doc/numpy-dev/reference/arrays.datetime.html>.
They made the decision to ignore leap seconds for simplicity reasons which
I feel rather strongly to be a poor choice and a bit of a copout. So I
would advocate handling leap seconds correctly in the Julia implementation
and otherwise being compatible.
2. A 128-bit timestamp with a similar design to the 64-bit one but
with much greater simultaneous range and precision.
3. Corresponding timedelta types.
The first step here is to understand the NumPy datetime64 design better.
Post by MilktraderThanks, this is a good start.
Dan
Post by Avik SenguptaThere have been some efforts towards dates and times
There is a very simple datetime class at https://github.com/aviks/**ju
**lia/blob/dates/base/date.jl<https://github.com/aviks/julia/blob/dates/base/date.jl>
and a more complex implementation is at https://github.com/**JeffreyS*
*arnoff/julia/tree/jtm/**extras/**jtm<https://github.com/JeffreySarnoff/julia/tree/jtm/extras/jtm>. However, all this is work in progress.
Regards
-
Avik
Post by MilktraderWhat work is being done to create a Time type?
I did find two time-related functions in base/libc.jl ...
julia> lastday = strptime("%Y-%m-%d","2012-12-**2**1")
1.356066e9
julia> typeof(lastday)
Float64
julia> strftime(lastday)
"Fri Dec 21 00:00:00 2012"
So presumably lastday is a Float64 representing seconds since Unix epoch.
My end goal is a type similar to zoo, xts in R, which is basically a
matrix indexed by time class vector.
Is anyone working on this and if not, what is a good place for me to start?
Regrads,
Dan
--
--
--
--
--