Migrate trackers from Sourceforge

Hi Fabien,

I'll begin work on this migration on Monday. The way this works is as follows:

1. I do an initial import to a local instance of Launchpad to verify the data we're getting from SourceForge is sane.
2. I clean up the data if necessary
3. We do a test import to a public, non-production instance of Launchpad so that you to can check the imported data.
4. If all goes well we do a full import to production.

The length of time this takes depends on how much work needs doing on the data (though it's usually minimal) and on the back and forth between us doing test imports and you checking them for errors.

Please contact me if you've any further questions.

We've started to do an import to staging for testing purposes: https://bugs.staging.launchpad.net/stellarium

We're running into some odd datetime parsing problems that I'm working to resolve; hopefully I should be able to get the import done by the end of tomorrow.

Just an update on the status of this import.

The importer seems to be choking on a datetime conversion for SF item 1669863. I'm working to resolve this (the import doesn't fail on my local machine but it does fail when run on staging, which is odd). I'll update this question as soon as I know more.

OK,
Thanks,
If the problem is only for the item 1669863, you can just discard it.
Fabien

2009/2/4 Fabien Chéreau <email address hidden>:
> Question #57552 on Launchpad itself changed:
> https://answers.edge.launchpad.net/launchpad/+question/57552
>
> Status: Answered => Open
>
> Fabien Chéreau is still having a problem:
> OK,
> Thanks,
> If the problem is only for the item 1669863, you can just discard it.
> Fabien

Okay. I'll try to fix the problem (I think there's an issue with
pytz.tzinfo.DstTzInfo that's causing it to choke, which could be bad
for other imports) but if that doesn't work I'll remove that item and
try continuing with the import.

We've managed to complete the import on staging: http://bugs.staging.launchpad.net/stellarium (when it's up; it's down at the moment, alas). There was a problem with our import script, which I've fixed.

Please take a look at the imported bugs and let me know if you see any obvious errors.

OK,
I'll let you know tomorrow morning (European time).
Fabien

Sadly, our latest staging restore failed spectacularly, so the database had to be rebuilt. I'll re-do the import once the database is back up and we'll try this again. Sorry for the delay.

Right, I was wondering :)
It's OK, the last thing I could see yesterday is that it seemed that some reports where not into LP, but maybe you didn't perform the full import.
Also are the attached files somehow imported also?
Fabien

2009/2/5 Fabien Chéreau <email address hidden>:
> Question #57552 on Launchpad itself changed:
> https://answers.edge.launchpad.net/launchpad/+question/57552
>
> Status: Solved => Open
>
> Fabien Chéreau is still having a problem:
> Right, I was wondering :)
> It's OK, the last thing I could see yesterday is that it seemed that some reports where not into LP, but maybe you didn't perform the full import.

Hmm. That depends when you looked. When I'd done the full import
staging wasn't responding, so I suspect you were looking whilst we
were still having problems with parsing some of the dates.

> Also are the attached files somehow imported also?

They should be, yes.

--
Graham Binns | PGP Key: 4DAD18FA

Hi Fabien,

We've re-imported the bugs to bugs.staging.launchpad.net/stellarium, so please take a look and make sure that the data we've imported look sane (I've spotted an issue with the dates on https://bugs.staging.launchpad.net/stellarium/+bug/329104 which we'll resolve before doing the import to production).

Once you're happy with the imported data we'll import the bugs to the production database.

Hi Fabien,

Just to keep you up-to-date with the import issues we were encountering.

I think I've isolated the problem (though I can't guarantee at this point that I've fixed it completely). The issue appears to be with the script that we use to export the bugs from SourceForge. For some reason it's skipping bugs - I think this is because it handles errors in an odd way. I've re-done the export and at least some of the bugs that were missing have now turned up, ready for importing. I'll re-run the import locally and then run it on staging, which will import all the bugs that haven't been imported whilst leaving the already-imported bugs as they are.

I'll hopefully get this done this evening some time. I'll update the question again when it's been done.

OK, I'll check the progress.
Fabien

Hi Fabien,

Sorry for not getting back to you earlier. As you can see the staging
database has once again been reverted, losing all the import work that
we had done.

I'm working on getting a standalone, empty Launchpad instance up for
us to test this on. I should have something by the end of tomorrow.
I'll keep you informed.

Cheers,

Graham