External datetime package synergy

[jrstrunk]

Is there anything I can do to Tempo to make using it with Pog more ergonomic? Would adding to_pog and from_pog functions to the Tempo library make sense? Should to_tuple and from_tuple functions be added for the date / time functions in Pog to make them easily usable with any other package? Should we just leave it alone and let users write their own middleware? I am open to any ideas. Relevant issue.

I honestly use SQLite mostly so Postgres date stuff is not something I am experienced with, but I know it is popular.

[lpil]

Hello! Yes I'd like to talk to you and @massivefermion about this.

There's a clear need for good time interop between all the packages of the ecosystem. I believe the approach we took with the HTTP libraries would work well here, having a gleam_* package which defines the core data structures and then have all other packages build upon it.

The problem we have is that the design of these data structures has not been decided on, and no one in core has had the time to dig into this. We need to come to some agreement to be able to move forward.

With my little bit of context I like the design of both Tempo and Birl. Tempo has an advantage in that it is easier to understand when printed, while Birl is more compact so performs better, and it doesn't permit invalid states the same way that Tempo does.

I'm not sure what the best way to proceed is here presently!

[jrstrunk]

Oh yes that makes the most sense! Do you want me to create a discussion in the gleam-lang/gleam repo for it so the community can weigh in? I understand this may be an open discussion for a while, and isn't something we should rush.

Also when you say Tempo permits invalid states, do you mean how datetime.literal("2024-11-15T04:13:00-04:00") and naive_datetime.literal("2024-11-15T04:13:00) are not comparable, or something else?

[lpil]

That sounds good!

For example, Date contains year, month, day ints, and you could have month 13 or day -1

[jrstrunk]

Okay I'll do that soonish! I might gather some info first though to make discussion starter more informed.

Oh, yes but the Date type is opaque so users can't construct it unless the values are first verified via a function like date.new. Are you worried that (1.) we can't have an opaque type in the std package or worried about (2.) there being some gap in the validation logic?

Also just for a little context :)
Birl's day type is defined as:

pub type Day {
  Day(year: Int, month: Int, date: Int)
}

while Tempo's Date is defined as:

pub type Month {
  Jan
  Feb
  Mar
  Apr
  May
  Jun
  Jul
  Aug
  Sep
  Oct
  Nov
  Dec
}

pub opaque type Date {
  Date(year: Int, month: Month, day: Int)
}

[massivefermion]

I think we should have a list of most probable use cases and try to come up with a good API for each one of them based on the existing experience. This way we can talk about it in a concrete way instead of an abstract way that may cause useless or unhelpful APIs.

[jrstrunk]

Good idea, even just looking at the Gleam ecosystem now we can see lots of examples of concrete use cases.

We should also establish a short term and long term goals for the gleam time std package. My interpretation is that a short term goal would be a simple package to provide a layer of interop between the different time packages in the ecosystem. Is this your vision Louis?

I see it as being between two opposing forces: getting it right (in a Gleamy way, which is going to take a lot of consideration), and shipping it soon (because the longer we wait the harder it will be to bring the ecosystem back together). I have a small proposal that may alleviate this tension:

Thinking about the time types, the DateTime type will probably take the most amount of consideration, while the Timestamp type (unix timestamp, just an int) will probably take the least. My proposal is to make creating a standard Timestamp type a high priority and release it soon, then let the ecosystem use that type as the basis for package interop as the more complex types are figured out and added to the std package later.

If we do this, all packages that deal with time can then be encouraged to provide functions to convert their types to and from the std Timestamp type. The pro to this is immediate relief to the time interop problem, while the con is introducing lots of to_timestamp and from_timestamp functions in every package that will probably be deprecated later. Though that con is probably mitigated because either way there will be lots of deprecations when the full std time package is released.

I think the (unix) Timestamp type is a good place to start because it is very simple, is well known, and can stand in for date, time, naive datetime, and UTC datetime types.

Thoughts?

[lpil]

My interpretation is that a short term goal would be a simple package to provide a layer of interop between the different time packages in the ecosystem. Is this your vision Louis?

Yes, spot on.

Thinking about the time types, the DateTime type will probably take the most amount of consideration, while the Timestamp type (unix timestamp, just an int) will probably take the least. My proposal is to make creating a standard Timestamp type a high priority and release it soon, then let the ecosystem use that type as the basis for package interop as the more complex types are figured out and added to the std package later.

I hadn't even considered having just a timestamp type! I've not thought about it much, but it could be a good idea.

What would be the purpose of this type in 5-10 years time, when a DateTime type exists? Are there still use-cases? Would having a time type that doesn't encode an offset or anything result in any sharp edges for future programmers?

[jrstrunk]

I think it's long term role would mainly be easy integration with external things that rely on timestamps. Something like SQLite is widely used but does not support date/time types: if you want to sort a SQLite query by date, you have to use a timestamp value in an Int column. If you wanted to implement a SQLite driver for Gleam (sqlight for example), you could add sqlight.timestamp and sqlight.decode_timestamp functions to both provide semantics to the field (as opposed to just sqlight.int) and to guard against pitfalls with timestamps (more on that later).

I think the biggest motivation is that timestamps are prominent enough that if we do not provide a type for them, users will still use them but just with an inferior Int type. Just speculation and some reflection on my career, but I think newer programmers tend to lean towards using timestamps instead of full datetime types because they are a little less daunting and you do not have to learn / think about offsets and conversions.

My biggest want for a timestamp type would just be some way to make sure that I am not accidentally comparing a second timestamp to a millisecond timestamp. Using Ints as timestamps I always have to double check documentation to determine if I need to do some sort of conversion first. Here is what a quick implementation could look like:

pub type Timestamp {
  Timestamp(seconds: Int)
  TimestampMilli(milliseconds: Int)
  TimestampMicro(microseconds: Int)
}

pub fn to_seconds(timestamp: Timestamp) -> Int {
  case timestamp {
    Timestamp(seconds:) -> seconds
    TimestampMilli(milliseconds:) -> milliseconds / 1000
    TimestampMicro(microseconds:) -> microseconds / 1_000_000
  }
}

pub fn to_milliseconds(timestamp: Timestamp) -> Int {
  case timestamp {
    Timestamp(seconds:) -> seconds * 1000
    TimestampMilli(milliseconds:) -> milliseconds
    TimestampMicro(microseconds:) -> microseconds / 1000
  }
}

pub fn to_microseconds(timestamp: Timestamp) -> Int {
  case timestamp {
    Timestamp(seconds:) -> seconds * 1_000_000
    TimestampMilli(milliseconds:) -> milliseconds * 1000
    TimestampMicro(microseconds:) -> microseconds
  }
}

This way you could always be sure you are comparing / using the right values, as there is no way to get the underlying int value without first acknowledging the precision in some way.

Timestamps definitely do have a sharp edge, but I think this is a case where it is up to the programmer to understand how to use the tools they have. That can be up for discussion though! I wanted to get your approval first Louis, but I would also want to open this discussion up to the community before publishing anything, just in case the three of us here miss something.

[lpil]

That design would mean that you cannot use == with timestamps or use them as keys in a dict or a set, which seems like it would be irritating.

What do you think of always using milliseconds as the timestamp precision?
I would imagine for further precision one would want something more designed for measuring durations and ordering, like an "Instant" type that would also be monotonic.

[jrstrunk]

Very good points! Yes I think having the type with just millisecond precision would be really nice. It would be a little less versatile, but I think that trade off would be worth it as you outlined. It could look something like:

pub type Timestamp {
  Timestamp(milliseconds: Int)
}

If we standardized the timestamp type to be milliseconds, then that would actually achieve almost as strong of guarantees as my first implementation, because if everyone uses the standard Timestamp type instead of an Int type, then I can be sure of the precision without having to rely on library specific documentation!

Do you want to include a couple simple helper functions in the standard package as well? Something like

pub fn to_seconds(timestamp: Timestamp) -> Int {
  timestamp.milliseconds / 1000
}

// Maybe, for use in pipelines mostly I assume
pub fn to_milliseconds(timestamp: Timestamp) -> Int {
  timestamp.milliseconds
}

It could be in the gleam/timestamp module of the gleam_time package.

[lpil]

Sounds good to me!

The Elm time package also has some functions for working with timestamps given a time zone. Do you think it would be useful to adopt any of them? Or do we want to give more thought to the time zone type? https://package.elm-lang.org/packages/elm/time/latest/Time

[jrstrunk]

Great!

Those Elm functions do seem useful, but I think the time zone type will take the most amount of consideration -- even more so than usual because the JS target makes things tricky. Given that functions like that can be added later without needing to change the underlying Timestamp type, I am inclined to release a standard type first to just allow interop, then work out the details of time zones over time.

Since we have some momentum in a direction, I will move this discussion over to a more publicly visible space so the community can weigh in. Unless you object you can close this issue.