Skip to content

Fix youtube CSV playlist importer - #4787

Merged
unixfox merged 5 commits into
iv-org:masterfrom
ThatMatrix:fix-youtube-playlist-importer
Feb 6, 2026
Merged

Fix youtube CSV playlist importer#4787
unixfox merged 5 commits into
iv-org:masterfrom
ThatMatrix:fix-youtube-playlist-importer

Conversation

@ThatMatrix

@ThatMatrix ThatMatrix commented Jul 11, 2024 •
edited
Loading

Copy link
Copy Markdown

This PR is a follow up of this issue #3954 .

tldr: This PR aims to fix the Youtube CSV Importer due to Google Takeout changing their export format

I've recently used a invidious account, exported all my playlists data from Google Takeout. However, it seems that they changed their format again by putting most of the datas about a playlist in playlists.csv. Only to leave the bare minimum in the playlist specific files.

This PR changes this by putting default values where info could be extracted before and fixing the out of bound error that could be happening depending on the playlist format.

Here is the error encountered:

Title: `Index out of bounds (IndexError)`
Date: `2024-07-11T01:36:38Z`
Route: `/data_control?referer=%2Ffeed%2Fplaylists`
Version: `2024.04.27-eda7444 @ master`

<details>
<summary>Backtrace</summary>
<p>
   
Index out of bounds (IndexError)
  from /usr/share/crystal/src/json/any.cr:237:3 in 'update_data_control'
  from lib/kemal/src/kemal/route.cr:12:9 in '->'
  from src/invidious/helpers/handlers.cr:30:37 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next'
  from lib/kemal/src/kemal/filter_handler.cr:21:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next'
  from src/invidious/helpers/handlers.cr:94:12 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next'
  from src/ext/kemal_static_file_handler.cr:112:11 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/request_processor.cr:51:11 in 'handle_client'
  from /usr/share/crystal/src/fiber.cr:146:11 in 'run'
  from ???

</p>
</details>

I also give you an extract of what a Google Takeout playlist file looks like

Video ID,Playlist video creation timestamp
dQw4w9WgXcQ,2024-06-24T23:49:51+00:00
-tJYN-eG1zk,2024-06-23T20:30:24+00:00

@ThatMatrix

ThatMatrix commented Jul 11, 2024 •
edited
Loading

Copy link
Copy Markdown
Author

I have tested importing a playlist with the final commit version and the provided docker-compose.yml file. It works with one of my GoogleTakeout playlist file.

@ThatMatrix

ThatMatrix commented Jul 11, 2024

Copy link
Copy Markdown
Author

I took the liberty to add a default value/placeholders for the description as well as privacy fields in the meantime but I would like to receive advice on what to do with them if possible.
Thanks a lot

@ThatMatrix
ThatMatrix marked this pull request as ready for review July 11, 2024 01:35
@ThatMatrix
ThatMatrix requested a review from a team as a code owner July 11, 2024 01:35
@ThatMatrix
ThatMatrix requested review from unixfox and removed request for a team July 11, 2024 01:35
@ThatMatrix ThatMatrix changed the title (DRAFT) Fix youtube CSV playlist importer Fix youtube CSV playlist importer Jul 11, 2024
@SamantazFox

SamantazFox commented Jul 18, 2024

Copy link
Copy Markdown
Member

Can you make a separate function for the new format?
Users who have exported when v1 was around should still be able to import their data!

@SamantazFox SamantazFox added the unfinished More work is needed on this PR, or on something this PR uses. label Jul 21, 2024
@PassiveLemon

PassiveLemon commented Sep 11, 2024

Copy link
Copy Markdown

Importing works for me with this PR

@jakkarth

jakkarth commented Jan 23, 2026

Copy link
Copy Markdown

Newly setting up Invidious, surprised that this bug and bug #4069. @SamantazFox or anyone what's the plan here? You've had a fix sitting here for 18 months. Hopefully no one still has v1 exports lying around, but even if they do, surely the default should be to support the current version? The perfect is the enemy of the good enough. I request that you merge this, and if v1 support is needed we can get the v1 import code from git history and add a way to support both versions at that time. But at least let current exports work, please.

@unixfox

unixfox commented Jan 23, 2026

Copy link
Copy Markdown
Member

I share the same vision as you @jakkarth. Supporting the old format shouldn't be a top priority, that could be a maintenance burner.

If this code still works fine, I'm good for merging it.

@jakkarth

jakkarth commented Jan 23, 2026

Copy link
Copy Markdown

I applied it this morning to master, built the image and ran it, imported all of my playlists without issue with the diff applied. About 800 videos across a dozen playlists successful.

@unixfox
unixfox merged commit ce94941 into iv-org:master Feb 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@unixfox unixfox Awaiting requested review from unixfox unixfox was automatically assigned from iv-org/developers

Assignees

No one assigned

Labels

unfinished More work is needed on this PR, or on something this PR uses.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@ThatMatrix @SamantazFox @PassiveLemon @jakkarth @unixfox