Editing Eurovision Wiki:Bots/Noticeboard (section)

== SQL files no longer being generated ==
{{Tracked|T416416}}
{{userlinks|The Anomebot2}}

I see that the Wikipedia dumps are now being generated using new infrastructure, and the old dumps have fallen into disrepair. Unfortunately, this includes the production of the SQL metadata files that contain the page-to-category mappings needed to drive my bot, so The Anomebot2 is now on indefinite hiatus again until this gets fixed. There is simply no way I can handle the entire XML dumps, with their terabytes of text, and I shouldn't need to anyway if the developers get their act together. I imagine I'm not the only bot operator in this situation. 

Can someone on the development team fix this? &mdash; [[User:The Anome|The Anome]] ([[User talk:The Anome|talk]]) 20:39, 3 February 2026 (UTC)

:You'd probably do better to file a request in [[phab:|Phabricator]] to get the attention of the development team. OTOH, if you want to give more information on which SQL dumps you're using, we might be able to figure out how to get that information out of the Toolforge database replicas (i.e. the same thing [[quarry:|Quarry]] uses). [[User:Anomie|Anomie]][[User talk:Anomie|⚔]] 21:51, 3 February 2026 (UTC)

::For private reasons, I don't use Phabricator. Can someone else file this for me? The specific SQL files I use are enwiki-[date]-page.sql.gz and enwiki-[date]-categorylinks.sql.gz, as can be found for example here:  https://dumps.wikimedia.org/enwiki/20260101/ &mdash; [[User:The Anome|The Anome]] ([[User talk:The Anome|talk]]) 22:11, 3 February 2026 (UTC)
:::I filed [[phab:T416416]]. What's the problem exactly? Are these files not getting created at all since a certain date? Are they getting created but their format changed and is omitting something you need? –[[User:Novem Linguae|<span style="color:blue">'''Novem Linguae'''</span>]] <small>([[User talk:Novem Linguae|talk]])</small> 22:55, 3 February 2026 (UTC)
:::If you want to try it out, I confirmed that both of those tables are available on the Toolforge replicas. [[User:Anomie|Anomie]][[User talk:Anomie|⚔]] 22:57, 3 February 2026 (UTC)
::::As are the dump files: cd to /public/dumps/public/enwiki/latest, and enwiki-latest-categorylinks.sql.gz is right there symlinked to ../20260201/enwiki-20260201-categorylinks.sql.gz and enwiki-latest-page.sql.gz to ../20260201/enwiki-20260201-page.sql.gz. —[[User:Cryptic|Cryptic]] 03:07, 4 February 2026 (UTC)
:::::And https://dumps.wikimedia.org/enwiki/20260201/ is sitting there with the (in-progress) February dumps as usual. :confused: —[[User:Cryptic|Cryptic]] 03:28, 4 February 2026 (UTC)
::::Thanks @[[User:Novem Linguae|Novem Linguae]] for bringing this to my attention.
::::@[[User:The Anome|The Anome]]: As noted on the [https://phabricator.wikimedia.org/T416416 phabricator] ticket, both of these dumps continue to be generated and are not deprecated. The schedule did change: we used to generate them twice per month (on the 1st and the 20th) and now we generate them only once (on the 1st). [[User:XCollazo-WMF|XCollazo-WMF]] ([[User talk:XCollazo-WMF|talk]]) 15:06, 4 February 2026 (UTC)
:@[[User:The Anome|The Anome]]: do you have a link to the announcement that some SQL dumps are going away? I hadn't seen anything. [[User:Legoktm|Legoktm]] ([[User talk:Legoktm|talk]]) 22:34, 3 February 2026 (UTC)
::Presumably a reference to [[phab:T414389]], which does say that only the XML dumps of the old flavor are eventually going to turn off. But maybe there is something else that The Anome has seen. [[User:Izno|Izno]] ([[User talk:Izno|talk]]) 00:31, 4 February 2026 (UTC)
:::Right, that's what I saw on the [[listarchive:list/xmldatadumps-l@lists.wikimedia.org/thread/E6D5EU4PMSTSOI2J7A46HJ3YW2W554CS/|mailing list]] too, but that announcement also said that SQL dumps were unaffected. [[User:Legoktm|Legoktm]] ([[User talk:Legoktm|talk]]) 00:38, 4 February 2026 (UTC)
::::I think the issue I saw was that the dump seemed to have stalled, and the file was not being displayed at that time. It's definitely displaying now, with a dump date of 2026-02-03 12:46:21. If we are still going to generate just the SQL dumps the old way, could they go back to being twice monthly, preferably on the 1st and 14th of each month? &mdash; [[User:The Anome|The Anome]] ([[User talk:The Anome|talk]]) 15:50, 4 February 2026 (UTC)
:::::TBH, I think a lot of people have provided suggestions and guidance on what you could do, but at this point it's on you to provide more details on what specifically you're doing, and why other alternatives aren't sufficient to persuade someone to figure out how to reinstate bimonthly dumps.
:::::I'm not sure if it needs to be explicitly stated, but the English Wikipedia keeps growing and it requires active work to scale up to maintain the status quo. I don't know the details that went into this specific decision, but it's at least understandable. [[User:Legoktm|Legoktm]] ([[User talk:Legoktm|talk]]) 22:08, 5 February 2026 (UTC)