[Scons-dev] Migrating issues from Tigris to GitHub
Bill Deegan
bill at baddogconsulting.com
Fri Jan 26 17:46:42 EST 2018
Is there any way to enumerate all the existing PR/Issues and overwrite only
the issues? (and/or items without corresponding changelists)?
That'd allow us to re-use the existing 3k or so aborted migrated issues.
-Bill
On Fri, Jan 26, 2018 at 1:34 PM, Andrew Featherstone <
andrew.featherstone at cantab.net> wrote:
> Hi All,
>
> I've addressed all the issues that I'm aware of with the importing of
> issues into GitHub. I don't kniow if there's anything we can do to fix the
> existing changes to the merge requests, but AFAICT we're ready to give this
> another go.
>
> Regards,
> Andrew
>
> On 3 January 2018 at 18:37, Bill Deegan <bill at baddogconsulting.com> wrote:
>
>> Crap.
>> It looks like the migration script overwrote existing pull requests.
>> They are all gone and replaced with migrated bugs.
>>
>> Let me see if I can get ahold of github and get them restored..
>>
>> -Bill
>>
>> On Tue, Jan 2, 2018 at 1:39 PM, Bill Deegan <bill at baddogconsulting.com>
>> wrote:
>>
>>> o.k. I'm going to disable the issues and remove the lock on the
>>> scons/scons repo for now. I'll fork out another for use trying out any
>>> fixes before rolling out to the main repo.
>>> -Bill
>>>
>>> On Tue, Jan 2, 2018 at 12:57 PM, Andrew Featherstone <
>>> andrew.featherstone at cantab.net> wrote:
>>>
>>>> Hi All,
>>>>
>>>> Yes you're right, it seems that "every pull request is an issue, but
>>>> not every issue is a pull request." as noted at
>>>> https://developer.github.com/v3/issues/#list-issues. I've raised a
>>>> couple of issues at https://github.com/SCons/tigris-to-github/issues
>>>> to submit my fixes against. I'm not sure at this point how to fix the
>>>> comments in the pull requests, but one step at a time.
>>>>
>>>> Andrew
>>>>
>>>> On 2 January 2018 at 19:11, Daniel Moody <dmoody256 at gmail.com> wrote:
>>>>
>>>>> Yes the 36 Pull Request we had got overwritten with the bug
>>>>> information for 1-36.
>>>>>
>>>>> If possible i think it would be best to move the pull request to the
>>>>> newest numbers so all the cross references to issue numbers can stay the
>>>>> same.
>>>>>
>>>>> Also another issue I saw was that # sign will link to other issues,
>>>>> which in some of the text of our issue, that was not intended. If possible
>>>>> you can replace '#[number]' with 'choice [number]' in the text when they
>>>>> get reprocessed?
>>>>>
>>>>> I read about choice here:
>>>>> https://github.com/github/markup/issues/303
>>>>>
>>>>> On Tue, Jan 2, 2018 at 12:03 PM, Bill Deegan <
>>>>> bill at baddogconsulting.com> wrote:
>>>>>
>>>>>> Andrew,
>>>>>>
>>>>>> Also.. there's 36 bugs which didn't get migrated.
>>>>>> I'm guessing they're the ones which yielded:
>>>>>> 422 {'message': 'Validation Failed', 'errors': [],
>>>>>> 'documentation_url': 'https://developer.github.com/
>>>>>> v3/issues/#edit-an-issue'}
>>>>>>
>>>>>> ...
>>>>>> 8
>>>>>> 422 {'message': 'Validation Failed', 'errors': [],
>>>>>> 'documentation_url': 'https://developer.github.com/
>>>>>> v3/issues/#edit-an-issue'}
>>>>>> 422 {'message': 'Validation Failed', 'errors': [],
>>>>>> 'documentation_url': 'https://developer.github.com/
>>>>>> v3/issues/#edit-an-issue'}
>>>>>> 422 {'message': 'Validation Failed', 'errors': [],
>>>>>> 'documentation_url': 'https://developer.github.com/
>>>>>> v3/issues/#edit-an-issue'}
>>>>>> 422 {'message': 'Validation Failed', 'errors': [],
>>>>>> 'documentation_url': 'https://developer.github.com/
>>>>>> v3/issues/#edit-an-issue'}
>>>>>> 422 {'message': 'Validation Failed', 'errors': [],
>>>>>> 'documentation_url': 'https://developer.github.com/
>>>>>> v3/issues/#edit-an-issue'}
>>>>>> 422 {'message': 'Validation Failed', 'errors': [],
>>>>>> 'documentation_url': 'https://developer.github.com/
>>>>>> v3/issues/#edit-an-issue'}
>>>>>> 422 {'message': 'Validation Failed', 'errors': [],
>>>>>> 'documentation_url': 'https://developer.github.com/
>>>>>> v3/issues/#edit-an-issue'}
>>>>>> 422 {'message': 'Validation Failed', 'errors': [],
>>>>>> 'documentation_url': 'https://developer.github.com/
>>>>>> v3/issues/#edit-an-issue'}
>>>>>> 422 {'message': 'Validation Failed', 'errors': [],
>>>>>> 'documentation_url': 'https://developer.github.com/
>>>>>> v3/issues/#edit-an-issue'}
>>>>>> 422 {'message': 'Validation Failed', 'errors': [],
>>>>>> 'documentation_url': 'https://developer.github.com/
>>>>>> v3/issues/#edit-an-issue'}
>>>>>> 9
>>>>>> ...
>>>>>>
>>>>>> So I'm guessing
>>>>>> Can you take a look at that?
>>>>>> For now I'm going to disable issues on scons/scons and then I'll try
>>>>>> again once you're pull request is ready.
>>>>>>
>>>>>> Also looks like the lowest issue # created is 37 (notable that the
>>>>>> highest pull request # is 36).
>>>>>> It looks like they share the same "namespace" for lack of a better
>>>>>> term.
>>>>>> So cross references to other bugs will be off..
>>>>>>
>>>>>> Thanks,
>>>>>> Bill
>>>>>>
>>>>>> On Tue, Jan 2, 2018 at 1:32 AM, Andrew Featherstone <
>>>>>> andrew.featherstone at cantab.net> wrote:
>>>>>>
>>>>>>> Hi Bill,
>>>>>>>
>>>>>>> I'm glad you persevered. No I've not put any effort into making this
>>>>>>> work with Python 2. There's some crude retry mechanisms in the
>>>>>>> implementation. By the look of https://github.com/scons/scons/issues
>>>>>>> this is still trundling on, as the issue count keeps going up when I
>>>>>>> refresh the browser.
>>>>>>>
>>>>>>> Unfortunately, I've spotted a meta-issue, which can be seen at
>>>>>>> https://github.com/SCons/scons/issues/1345, for example. There's an
>>>>>>> attachment for this issue that should be at
>>>>>>> https://github.com/scons/tigris-issue-attachments/blob/maste
>>>>>>> r/45/SCons-PkgCheckModules.diff, but is pointing to
>>>>>>> https://github.com/bdbaddog/scons/tigris-issue-attachments/b
>>>>>>> lob/master/45/SCons-PkgCheckModules.diff instead.
>>>>>>>
>>>>>>> I think the sensible thing is for me to raise an issue against
>>>>>>> https://github.com/SCons/tigris-to-github and submit a fix via a
>>>>>>> pull request. The import process will merrily overwrite existing issues so
>>>>>>> that's not a problem, but we will need to run the script again.
>>>>>>>
>>>>>>> Sorry, all.
>>>>>>>
>>>>>>> Andrew
>>>>>>>
>>>>>>>
>>>>>>> On 2 January 2018 at 04:48, Bill Deegan <bill at baddogconsulting.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Got past that. Requires python 3 and a few modules.
>>>>>>>> Running now but just got the folllowing:
>>>>>>>> GitHub repository for issues: scons/scons
>>>>>>>> GitHub repository for attachments: scons/tigris-issue-attachments
>>>>>>>> 1
>>>>>>>> 2
>>>>>>>> 3
>>>>>>>> 4
>>>>>>>> 5
>>>>>>>> 6
>>>>>>>> 7
>>>>>>>> 8
>>>>>>>> 422 {'message': 'Validation Failed', 'errors': [],
>>>>>>>> 'documentation_url': 'https://developer.github.com/
>>>>>>>> v3/issues/#edit-an-issue'}
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Jan 1, 2018 at 8:05 PM, Bill Deegan <
>>>>>>>> bill at baddogconsulting.com> wrote:
>>>>>>>>
>>>>>>>>> No go.
>>>>>>>>>
>>>>>>>>> python tigris2github.py
>>>>>>>>> Traceback (most recent call last):
>>>>>>>>> File "tigris2github.py", line 11, in <module>
>>>>>>>>> from github import Github, UnknownObjectException
>>>>>>>>> ImportError: No module named github
>>>>>>>>>
>>>>>>>>> pip search github yields a long list.
>>>>>>>>>
>>>>>>>>> Can you add a requirements.txt with a list of required packages to
>>>>>>>>> the repo and send a pull request?
>>>>>>>>>
>>>>>>>>> On Fri, Dec 29, 2017 at 3:37 AM, Andrew Featherstone <
>>>>>>>>> andrew.featherstone at cantab.net> wrote:
>>>>>>>>>
>>>>>>>>>> I've just check this morning that it's able to work with an
>>>>>>>>>> organization's repos and not just an individual's. If you don't want to
>>>>>>>>>> solicit feedback on the users' mailing list, then the next step is to
>>>>>>>>>> perform the migration. So, to move things forward:
>>>>>>>>>>
>>>>>>>>>> 1. Clone my repo.
>>>>>>>>>> 2. Call python tigris2github.py and answer the questions.
>>>>>>>>>>
>>>>>>>>>> So it'd be something like this:
>>>>>>>>>>
>>>>>>>>>> > python .\tigris2github.py
>>>>>>>>>> GitHub username: ajf58
>>>>>>>>>> GitHub Password:
>>>>>>>>>> GitHub repository for issues: scons/scons
>>>>>>>>>> GitHub repository for attachments: scons/tigris-issue-attachments
>>>>>>>>>>
>>>>>>>>>> As mentioned previously, it'll take a couple of hours to churn
>>>>>>>>>> through everything due to the rate limiting of GitHub.
>>>>>>>>>>
>>>>>>>>>> Cheers,
>>>>>>>>>> Andrew
>>>>>>>>>>
>>>>>>>>>> On 28 December 2017 at 22:12, Bill Deegan <
>>>>>>>>>> bill at baddogconsulting.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Yes. Sorry I didn't see the link in the existing markup.
>>>>>>>>>>> That looks good to me.
>>>>>>>>>>> So to move forward (migrate bugs to scons/scons and attachments
>>>>>>>>>>> to scons/tigris-issue-attachements/ ) what do I need to do?
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Bill
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Dec 28, 2017 at 12:56 PM, Andrew Featherstone <
>>>>>>>>>>> andrew.featherstone at cantab.net> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Bill,
>>>>>>>>>>>>
>>>>>>>>>>>> I'm not sure I follow you. The issue numbers and the Tigris
>>>>>>>>>>>> 'attachment IDs' (used as the folder names) aren't the same value. Where
>>>>>>>>>>>> issues have attachments they are reffered to in the issue by name, and the
>>>>>>>>>>>> URL is included in the Markdown. As an example, see
>>>>>>>>>>>> https://github.com/ajf58/tigris-sandbox/issues/6, which has an
>>>>>>>>>>>> attachment found at https://github.com/ajf58/tigri
>>>>>>>>>>>> s-issue-attachments/blob/master/190/bug6.zip.
>>>>>>>>>>>>
>>>>>>>>>>>> Have I understood your question correctly?
>>>>>>>>>>>>
>>>>>>>>>>>> Andrew
>>>>>>>>>>>>
>>>>>>>>>>>> On 28 December 2017 at 20:33, Bill Deegan <
>>>>>>>>>>>> bill at baddogconsulting.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Can you add to the github bug a link to the github attachment
>>>>>>>>>>>>> repo where appropriate?
>>>>>>>>>>>>> (so you don't have to go look for the repo and then the issue
>>>>>>>>>>>>> number folder.)
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Dec 28, 2017 at 11:51 AM, Daniel Moody <
>>>>>>>>>>>>> dmoody256 at gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> I think it's awesome, looks great fully support moving
>>>>>>>>>>>>>> forward with this! Good work!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Dec 28, 2017 2:36 PM, "Andrew Featherstone" <
>>>>>>>>>>>>>> andrew.featherstone at cantab.net> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Any comments or feedback on this?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>> Andrew
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 17 December 2017 at 18:58, Bill Deegan <
>>>>>>>>>>>>>>> bill at baddogconsulting.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Great work!
>>>>>>>>>>>>>>>> I'll try and set aside some time to review it this week.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> -Bill
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Sat, Dec 16, 2017 at 6:44 PM, Andrew Featherstone <
>>>>>>>>>>>>>>>> andrew.featherstone at cantab.net> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I've been working on how we can migrate the Tigris issues
>>>>>>>>>>>>>>>>> to GitHub. You can see the output of this at
>>>>>>>>>>>>>>>>> https://github.com/ajf58/tigris-sandbox/issues, To ensure
>>>>>>>>>>>>>>>>> that we're not relying on issue attachments being hosted at Tigris, which
>>>>>>>>>>>>>>>>> looks to be deserted by its owners (no tweets since 2013
>>>>>>>>>>>>>>>>> https://twitter.com/tigrisdotorg?lang=en, for example),
>>>>>>>>>>>>>>>>> the issue attachments are also migrated to GitHub (
>>>>>>>>>>>>>>>>> https://github.com/ajf58/tigris-issue-attachments). The
>>>>>>>>>>>>>>>>> source code used for doing this can be found at
>>>>>>>>>>>>>>>>> https://github.com/ajf58/tigris-to-github.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I think the next steps are:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 1. Members of this list review the format of the
>>>>>>>>>>>>>>>>> migrated issues and offer feedback.
>>>>>>>>>>>>>>>>> 2. Act on feedback.
>>>>>>>>>>>>>>>>> 3. Get feedback on the scons-user mailing list.
>>>>>>>>>>>>>>>>> 4. Migrate issues to https://github.com/SCons/scons
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> The migration process takes ~2 hours currently. This is
>>>>>>>>>>>>>>>>> largely due to the rate limits GitHub imposees on their REST API. GitHub
>>>>>>>>>>>>>>>>> allows temporary restrictions to be imposed on the repo, so we can do that
>>>>>>>>>>>>>>>>> while the migration takes place.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thoughts?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>>> Andrew
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>> Scons-dev mailing list
>>>>>>>>>>>>>>>>> Scons-dev at scons.org
>>>>>>>>>>>>>>>>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>> Scons-dev mailing list
>>>>>>>>>>>>>>>> Scons-dev at scons.org
>>>>>>>>>>>>>>>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>> Scons-dev mailing list
>>>>>>>>>>>>>>> Scons-dev at scons.org
>>>>>>>>>>>>>>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>> Scons-dev mailing list
>>>>>>>>>>>>>> Scons-dev at scons.org
>>>>>>>>>>>>>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>> Scons-dev mailing list
>>>>>>>>>>>>> Scons-dev at scons.org
>>>>>>>>>>>>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> Scons-dev mailing list
>>>>>>>>>>>> Scons-dev at scons.org
>>>>>>>>>>>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Scons-dev mailing list
>>>>>>>>>>> Scons-dev at scons.org
>>>>>>>>>>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Scons-dev mailing list
>>>>>>>>>> Scons-dev at scons.org
>>>>>>>>>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Scons-dev mailing list
>>>>>>>> Scons-dev at scons.org
>>>>>>>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Scons-dev mailing list
>>>>>>> Scons-dev at scons.org
>>>>>>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Scons-dev mailing list
>>>>>> Scons-dev at scons.org
>>>>>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Scons-dev mailing list
>>>>> Scons-dev at scons.org
>>>>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> Scons-dev mailing list
>>>> Scons-dev at scons.org
>>>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>>>
>>>>
>>>
>>
>> _______________________________________________
>> Scons-dev mailing list
>> Scons-dev at scons.org
>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>
>>
>
> _______________________________________________
> Scons-dev mailing list
> Scons-dev at scons.org
> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist2.pair.net/pipermail/scons-dev/attachments/20180126/dde3a8f2/attachment-0001.html>
More information about the Scons-dev
mailing list