From 5191da21db926ef063065ae3f9d306b4d948614a Mon Sep 17 00:00:00 2001 From: Andy <88590076+AAndyProgram@users.noreply.github.com> Date: Tue, 25 Nov 2025 12:58:23 +0300 Subject: [PATCH] 2025.11.25.0 --- Home.md | 3 ++ Settings.md | 82 ++++++++++++++++++++++++++++++++++++++++++++++------- 2 files changed, 74 insertions(+), 11 deletions(-) diff --git a/Home.md b/Home.md index 5bf383f..48d6127 100644 --- a/Home.md +++ b/Home.md @@ -204,6 +204,9 @@ This button looks like a bookmark. When you click on this button, a form for dow You may see a message like `Host [Site] is not ready` or `Host [Site] is unavailable`. This means that your credentials have expired, or some of your credentials are missing, or a saved posts for a particular site cannot be downloaded at this time. +Hot-keys: +- `Esc` - close the form + ![Saved posts window](https://raw.githubusercontent.com/AAndyProgram/SCrawler/main/ProgramScreenshots/SavedPosts.png) ### Download diff --git a/Settings.md b/Settings.md index b0a0159..0b61167 100644 --- a/Settings.md +++ b/Settings.md @@ -243,6 +243,7 @@ Version **2025.09.05** ([release](https://github.com/yt-dlp/yt-dlp/releases/tag/ - [YouTube](#youtube) - [ThisVid](#thisvid) - [TikTok](#tiktok) +- [xHamster](#xhamster) ### OF-Scraper +[See also site default fields](#site-default-fields) + ![Reddit settings](https://raw.githubusercontent.com/AAndyProgram/SCrawler/main/ProgramScreenshots/SettingsSiteReddit.png) ### Reddit requirements @@ -339,6 +339,8 @@ The Reddit parser can parse data without cookies, but you can add it if you like - `Concurrent downloads` - the number of concurrent downloads. - `Use the new Twitter icon (X)` - *restart SCrawler to take effect* +[See also site default fields](#site-default-fields) + ![Twitter settings](https://raw.githubusercontent.com/AAndyProgram/SCrawler/main/ProgramScreenshots/SettingsSiteTwitter.png) ### Twitter requirements @@ -374,6 +376,8 @@ To completely download a large profile, you can use sleep timers. The sleep time - `User name`- your account username (handle) - `Password` - your account password +[See also site default fields](#site-default-fields) + ![Bluesky settings](https://raw.githubusercontent.com/AAndyProgram/SCrawler/main/ProgramScreenshots/SettingsSiteBluesky.png) ### Bluesky requirements @@ -397,7 +401,9 @@ To completely download a large profile, you can use sleep timers. The sleep time - `keydb_api` - change the keydb_api. **Change this value only if you know what you are doing.** - `Private key` - path to the DRM key file `private_key.pem` - `Client ID` - path to the DRM key file `client_id.bin` - - `backend` - the value of `advanced_options` in the configuration. If you can't download the video, try using 'httpx'. + - `backend` - the value of `advanced_options` in the configuration. If you can't download the video, try using `httpx`. + +[See also site default fields](#site-default-fields) ![OnlyFans settings](https://raw.githubusercontent.com/AAndyProgram/SCrawler/main/ProgramScreenshots/SettingsSiteOnlyFans.png) @@ -519,6 +525,8 @@ To support the downloading of this site you should add the Mastodon domain to th **If an error occurred and the log told you that you need to update your credentials, you also need to enable the disabled functions (`Download timeline`, `Download stories`) back.** +[See also site default fields](#site-default-fields) + ![Instagram settings](https://raw.githubusercontent.com/AAndyProgram/SCrawler/main/ProgramScreenshots/SettingsSiteInstagram.png) ### Instagram data means @@ -635,6 +643,8 @@ result_cancel-->[*] - Download - `Download data` - The internal value indicates that site data should be downloaded. It becomes unchecked when the site returns an error. When you update your credentials, you need to checked this check box. +[See also site default fields](#site-default-fields) + ![Threads settings](https://raw.githubusercontent.com/AAndyProgram/SCrawler/main/ProgramScreenshots/SettingsSiteThreads.png) ### Threads requirements @@ -658,6 +668,8 @@ The `Accept` and `UserAgent` headers must be obtained from the page `https://fac If you can't find `x-ig-app-id` or `x-asbd-id`, you can get them from Instagram. +[See also site default fields](#site-default-fields) + ![Facebook settings](https://raw.githubusercontent.com/AAndyProgram/SCrawler/main/ProgramScreenshots/SettingsSiteFacebook.png) ### Facebook requirements @@ -678,6 +690,8 @@ The `User ID` and `UserAgent` values must be obtained from the page `https://jus **How to find [headers and values](#how-to-find-headers)** +[See also site default fields](#site-default-fields) + ![JustForFans settings](https://raw.githubusercontent.com/AAndyProgram/SCrawler/main/ProgramScreenshots/SettingsSiteJustForFans.png) ### JustForFans requirements @@ -701,6 +715,8 @@ The `User ID` and `UserAgent` values must be obtained from the page `https://jus - `UserAgent` - UserAgent to use in requests ([how to find](#how-to-find-useragent)) --> +[See also site default fields](#site-default-fields) + ![RedGifs settings](https://raw.githubusercontent.com/AAndyProgram/SCrawler/main/ProgramScreenshots/SettingsSiteRedGifs.png) ### RedGifs requirements @@ -740,12 +756,15 @@ The `User ID` and `UserAgent` values must be obtained from the page `https://jus See additional settings [here](https://github.com/AAndyProgram/SCrawler/wiki/YouTube-downloader#Settings). +[See also site default fields](#site-default-fields) + ![YouTube settings](https://raw.githubusercontent.com/AAndyProgram/SCrawler/main/ProgramScreenshots/SettingsSiteYouTube.png) ### YouTube requirements - [`Cookies`](#how-to-set-up-cookies) for private data - [**yt-dlp**](#YT-DLP) - [**ffmpeg**](#ffmpeg) +- [**Deno**](#deno) ### YouTube operational API To download communities, you need to install the `YouTube operational API`. @@ -781,10 +800,33 @@ cd C:\wamp64\www\YouTube-operational-API\ git pull https://github.com/Benjamin-Loison/YouTube-operational-API ``` +### Deno +**With the release of yt-dlp version `2025.11.12`, external JavaScript runtime support has arrived.** +**All users who intend to use yt-dlp with YouTube are strongly encouraged to install one of the supported JS runtimes.** + +The following JavaScript runtimes are currently supported (_in order of recommendation, from strongest to weakest_): + +- **recommended for most users** +- https://deno.com/ +- https://github.com/denoland/deno + - note: *if downloading from [Deno's GitHub releases](https://github.com/denoland/deno/releases/latest), get `deno` **not** `denort`* +- minimum Deno version supported by yt-dlp: `2.0.0` + - *the latest version of Deno is strongly recommended* + +**You can find full information [here](https://github.com/yt-dlp/yt-dlp/issues/15012)** + +#### Install Deno +Just launch PowerShell and run the following command: +```PowerShell +irm https://deno.land/install.ps1 | iex +``` + ## Pinterest - `Concurrent downloads` - the number of concurrent downloads. - `Saved posts user` - personal profile username. +[See also site default fields](#site-default-fields) + ![Pinterest settings](https://raw.githubusercontent.com/AAndyProgram/SCrawler/main/ProgramScreenshots/SettingsSitePinterest.png) ### Pinterest requirements @@ -802,6 +844,8 @@ git pull https://github.com/Benjamin-Loison/YouTube-operational-API - `Use video date as file date` - set the file date to the date the video was added (website) (if available) - `Use video date as file date (standalone downloader)` - set the file date to the date the video was added (website) (if available) +[See also site default fields](#site-default-fields) + ![TikTok settings](https://raw.githubusercontent.com/AAndyProgram/SCrawler/main/ProgramScreenshots/SettingsSiteTikTok.png) ### TikTok requirements @@ -841,6 +885,8 @@ if exist %p% ( - `Download GIFs as mp4` - download gifs in 'mp4' format instead of native 'webm'. - `Saved posts user` - personal profile username (to download saved posts) +[See also site default fields](#site-default-fields) + ![PornHub settings](https://raw.githubusercontent.com/AAndyProgram/SCrawler/main/ProgramScreenshots/SettingsSitePornHub.png) ### PornHub requirements @@ -859,9 +905,12 @@ About photos. Photo download problems have the lowest priority. You can still cr - `Download UHD` - Download UHD (4K) content - `Re-encode downloaded videos if necessary` - If enabled and the video is downloaded in a non-native format, the video will be re-encoded. **Attention! Enabling this setting results in maximum CPU usage.** +[See also site default fields](#site-default-fields) + ![XHamster settings](https://raw.githubusercontent.com/AAndyProgram/SCrawler/main/ProgramScreenshots/SettingsSiteXHamster.png) ### XHamster requirements +- [yt-dlp](#YT-DLP) - [ffmpeg](#ffmpeg) is required anyway - [`Cookies`](#how-to-set-up-cookies) for downloading saved posts @@ -869,6 +918,8 @@ About photos. Photo download problems have the lowest priority. You can still cr - `Download UHD` - Download UHD (4K) content. - `Playlist of saved videos` - Your personal videos playlist to download as 'saved posts'. This playlist must be private (Visibility = `Only me`). It also required cookies. This playlist must be entered by pattern: `https://www.xvideos.com/favorite/01234567/playlistname`. +[See also site default fields](#site-default-fields) + ![XVIDEOS settings](https://raw.githubusercontent.com/AAndyProgram/SCrawler/main/ProgramScreenshots/SettingsSiteXvideos.png) ### XVIDEOS requirements @@ -882,6 +933,8 @@ About photos. Photo download problems have the lowest priority. You can still cr - `Favourite videos` - download favourite videos. - `Different folders` - Use different folders to store video files. If `true`, then public videos will be stored in the `Public` folder, private - in the `Private` folder. If `false`, all videos will be stored in the `Video` folder. +[See also site default fields](#site-default-fields) + ![ThisVid settings](https://raw.githubusercontent.com/AAndyProgram/SCrawler/main/ProgramScreenshots/SettingsSiteThisVid.png) ### ThisVid requirements @@ -900,6 +953,8 @@ About photos. Photo download problems have the lowest priority. You can still cr ## LPSG +[See also site default fields](#site-default-fields) + ![LPSG settings](https://raw.githubusercontent.com/AAndyProgram/SCrawler/main/ProgramScreenshots/SettingsSiteLPSG.png) ### LPSG requirements @@ -942,6 +997,9 @@ It is very easy to create a new group. Just click the `Add a new download group` - `Sites` - You can select sites. - `Pencil` button for selected sites. Only users from the sites you have selected will be downloaded. - `Red X` button for excluded sites. Users from the sites you have selected will be excluded from the download. +- `Groups` - You can select groups/ + - `Pencil` edit groups. The groups you selected will **also** be downloaded + - `Left checkbox` - If checked, only the selected groups will be downloaded. All other options will be ignored. If unchecked: all the options of the new group will be used to filter the users to be downloaded; the selected groups will be also downloaded "as is", using their options. You can set both selected and excluded options for each selection. @@ -978,9 +1036,8 @@ You can set up automatic downloads. You can find these settings in `Settings - A ![Scheduler](https://raw.githubusercontent.com/AAndyProgram/SCrawler/main/ProgramScreenshots/SettingsScheduler.png) **Plan**: -- `Disable` - Disable automatic downloads. -- `Specified` - Specify custom download options (read more [here](#download-groups)). -- `Groups` - Automatic downloading of one or more existing groups (read more [here](#download-groups)). +- `Disabled` - Disable automatic downloads. +- `Enabled` - Enable automatic downloads (read more [here](#download-groups)). - `Show notifications` - Show notification when there is new downloaded data. - `Simple` - Show a simple notification instead of a user notification. This means that if any user data has been downloaded with the plan, a simple notification will be shown with the number of users downloaded. The 'Image' and 'User icon' parameters will be ignored. - `Image` - Show downloaded image in notification. @@ -1002,7 +1059,7 @@ This command can be a batch command or any script file (bat, ps1 or whatever you # How to set up cookies ## First method -1. Using a browser extension to receive site cookies, copy the cookies (or save them to a file) (I use this to get cookies: [Official site](https://www.editthiscookie.com/blog/2014/03/install-editthiscookie/), [Chrome Web Store](https://chromewebstore.google.com/detail/editthiscookie/fngmhnnpilhplaeedifhccceomclgfbg)). +1. Using a browser extension to receive site cookies, copy the cookies (or save them to a file) (you can use one of the [recommended extensions](#list-of-cookie-editing-extensions) or any other). 1. Open the settings of the site you want to configure (for example, `Main window` - `Settings` - `Instagram`). 1. Find the `Cookies` field. 1. Click on the `Pencil` button. @@ -1010,7 +1067,10 @@ This command can be a batch command or any script file (bat, ps1 or whatever you 1. Paste the copied cookies text into the window that opens (or press `Ctrl+O` to load cookies from a saved file (if you saved cookies to a file)) and click `OK`. 1. Click OK to close the cookies editor and save the cookies. -**Another similar extension: [Official site](https://cookie-editor.com/), [Chrome Web Store](https://chromewebstore.google.com/detail/cookie-editor/hlkenndednhfkekhgcdicdfddnkalmdm)** +### List of cookie editing extensions +- EditThisCookie ❌ - [Official site](https://www.editthiscookie.com/blog/2014/03/install-editthiscookie/), [Chrome Web Store](https://chromewebstore.google.com/detail/editthiscookie/fngmhnnpilhplaeedifhccceomclgfbg), [GitHub](https://github.com/fcapano/Edit-This-Cookie) +- **EditThisCookie v3** ✔️ - *(EditThisCookie fork)* [Official site](https://editcookie.com/), [GitHub](https://github.com/mrfanii/Edit-This-Cookie-V3), [Chrome Web Store](https://chromewebstore.google.com/detail/editthiscookie-v3/ojfebgpkimhlhcblbalbfjblapadhbol) +- **Cookie-Editor** ✔️ - [Official site](https://cookie-editor.com/), [GitHub](https://github.com/moustachauve/cookie-editor), [Chrome Web Store](https://chromewebstore.google.com/detail/cookie-editor/hlkenndednhfkekhgcdicdfddnkalmdm) ## Second method 1. Open Google Chrome, Microsoft Edge or FireFox.