Skip to content
This repository was archived by the owner on Apr 14, 2021. It is now read-only.

Update CLDR data to v27.0.3 #115

Merged
merged 4 commits into from
Jun 30, 2015
Merged

Update CLDR data to v27.0.3 #115

merged 4 commits into from
Jun 30, 2015

Conversation

Turbo87
Copy link
Contributor

@Turbo87 Turbo87 commented Jun 25, 2015

This PR is updating the CLDR data to v27.0.3 and also moves from downloading the data via grunt/CURL to using the appropriate NPM packages instead. The test suite seems to run fine, but this definitely should be checked by another pair of eyes before being merged.

@Turbo87
Remove unused "grunt-extract-cldr-data" dependency
fbfdc50 
This wasn't configured properly and didn't work
@caridy caridy self-assigned this Jun 25, 2015
@caridy
Copy link
Collaborator

caridy commented Jun 25, 2015

@Turbo87, thanks for putting this together. I tried to update when 27 was out, but there were so many issues, that I decided to deferred it. It seems that 27.0.3 is better. Nevertheless, I ran the same program I created to compute the diff when I was trying to upgrade, and I these is the result:

new locale added:  en-Shaw
new locale added:  iu-Latn
new locale added:  sw-CD
----------------------------------
old locale removed:  aa-DJ
old locale removed:  aa-ER
old locale removed:  aa-ET
old locale removed:  aa
old locale removed:  ia-FR
old locale removed:  ia
old locale removed:  nr-ZA
old locale removed:  nr
old locale removed:  nso-ZA
old locale removed:  nso
old locale removed:  ss-SZ
old locale removed:  ss-ZA
old locale removed:  ss
old locale removed:  ssy-ER
old locale removed:  ssy
old locale removed:  swc-CD
old locale removed:  swc
old locale removed:  tn-BW
old locale removed:  tn-ZA
old locale removed:  tn
old locale removed:  ts-ZA
old locale removed:  ts
old locale removed:  ve-ZA
old locale removed:  ve
old locale removed:  vo-001
old locale removed:  vo
------------------------------
Total preserved:  692
Total added:  3
Total removed:  26 (11 top level locales)

At the first glance, it seems to be ok. My only concern are those 11 top level locales that we will be eliminated. Ideally, no top level locale should be removed from version to version, maybe we should ask @rxaviers or @srl295 if they know why is that data removed.

@rxaviers
Copy link

cc @JCEmmons

@Turbo87
Copy link
Contributor Author

Turbo87 commented Jun 26, 2015

@caridy I fixed the second commit and regenerated the locale-data folder again

@caridy
Copy link
Collaborator

caridy commented Jun 26, 2015

@Turbo87 everything looks good now.

Let's way for @JCEmmons to see if he can give us more details about the following missing locales from CLDR 27.0.3:

old locale removed:  aa-DJ
old locale removed:  aa-ER
old locale removed:  aa-ET
old locale removed:  aa
old locale removed:  ia-FR
old locale removed:  ia
old locale removed:  nr-ZA
old locale removed:  nr
old locale removed:  nso-ZA
old locale removed:  nso
old locale removed:  ss-SZ
old locale removed:  ss-ZA
old locale removed:  ss
old locale removed:  ssy-ER
old locale removed:  ssy
old locale removed:  swc-CD
old locale removed:  swc
old locale removed:  tn-BW
old locale removed:  tn-ZA
old locale removed:  tn
old locale removed:  ts-ZA
old locale removed:  ts
old locale removed:  ve-ZA
old locale removed:  ve
old locale removed:  vo-001
old locale removed:  vo

@JCEmmons
Copy link

The removed locales were the result of a review on our part of CLDR locales that had very preliminary data submitted many years ago (like before I came to the project 10 years ago), but have not had any meaningful work done on them since. We have a "seed" repository that we use to keep locale data for locales that are deemed by the committee to be incomplete or perhaps inaccurate, and the locales on this list were moved from "common" to "seed". It is not very often that we do this, and we are in the process of trying to craft a more precise policy as to what goes into "common" vs. "seed". Refer to http://unicode.org/cldr/trac/changeset/11339

I don't expect to be moving any more locales out of "common" at any point in the near future.

@caridy
Copy link
Collaborator

caridy commented Jun 26, 2015

great, thanks @JCEmmons.

@Turbo87 I will get this merge shortly.

caridy added a commit that referenced this pull request Jun 30, 2015
@caridy
Merge pull request #115 from Turbo87/cldr-update
d8fa0eb 
PR #115: Update CLDR data to v27.0.3
@caridy caridy merged commit d8fa0eb into andyearnshaw:master Jun 30, 2015
@caridy
Copy link
Collaborator

caridy commented Jun 30, 2015

@Turbo87 I have merged the PR, but more work needs to be done. I noticed that the new data is missing a bunch of calendars:

>             "buddhist",
>             "chinese",
>             "coptic",
>             "dangi",
>             "ethioaa",
>             "ethiopic",
>             "generic",
>             "hebrew",
>             "indian",
>             "islamic",
>             "islamicc",
>             "japanese",
>             "persian",
>             "roc"

In fact, since it only uses cldr-dates-full pkg, it only produces the generic and gregorian, which is not enough. I totally forgot about that, my mistake for not spotting that during the review, since I used those when I tried 27.0.0, I will try to stash that part of the work later today.

caridy added a commit that referenced this pull request Jun 30, 2015
@caridy
follow up on PR #115: adding missing calendar packages from cldr
234cca8
caridy added a commit that referenced this pull request Jun 30, 2015
@caridy
getting 1.0.0 ready after PR #115
5e34203
@caridy
Copy link
Collaborator

caridy commented Jun 30, 2015

@Turbo87 new calendars were added in 234cca8

@caridy
Copy link
Collaborator

caridy commented Jun 30, 2015

I ran some validation on the new data, and it seems to be ok, this is a testament of it:

$ diff locale-data/json/en.json old-json/en.json
39,40d38
<                 "Hmsv": "HH:mm:ss v",
<                 "Hmv": "HH:mm v",
51,52d48
<                 "hmsv": "h:mm:ss a v",
<                 "hmv": "h:mm a v",

@andyearnshaw I think we are ready to roll out 1.0.0

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers
No reviews
Assignees

@caridy caridy

Labels
enhancement
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@Turbo87 @caridy @rxaviers @JCEmmons