diff --git a/Home.md b/Home.md index 3deb86c..5a68452 100644 --- a/Home.md +++ b/Home.md @@ -524,11 +524,7 @@ The program implements two ways to add users to this list: # Move the SCrawler data to a new location. 1. Close SCrawler. 1. Move the data to a new location. -1. In the SCrawler settings folder, find the `Settings.xml` file. - 1. Change the value of the `GlobalPath` node. - 1. For each site in this file (under the `Sites` node) where it exists, change the value of the `Path` node to the new location. Example: `Sites` - `Instagram` - `Path`. - 1. Save this file and close. -1. Open the `Users.xml` file (in the SCrawler settings folder). - 1. Check the `SpecialPath` values to make sure they still point to the correct locations. If not, replace these values with the correct ones. To find all non-empty `SpecialPath` values, you can use a simple regular expression: `SpecialPath="[^"]+"`. - 1. Save this file and close. +1. In the SCrawler settings folder, find the `Settings.xml` file. Change the value of the `GlobalPath` node. +1. Open each `Host_..._..._Name.xml` and `Responser_..._Settings.xml` file and update the `Path` and `SavedPostsPath` values to the new locations. +1. Open the `Users.xml` file (in the SCrawler settings folder). Check the `SpecialPath` and `SpecialCollectionPath` values to make sure they still point to the correct locations. If not, replace these values with the correct ones. To find all non-empty `SpecialPath` and `SpecialCollectionPath` values, you can use a simple regular expressions: `SpecialPath="[^"]+"`, `SpecialCollectionPath="[^"]+"`. 1. Open SCrawler. \ No newline at end of file diff --git a/Settings.md b/Settings.md index c510e0e..3315328 100644 --- a/Settings.md +++ b/Settings.md @@ -217,7 +217,7 @@ ffmpeg is required for several sites like Reddit, PornHub, XHamster, XVIDEOS, et ### Gallery-dl -Version **1.27.7** ([release](https://github.com/mikf/gallery-dl/releases/tag/v1.27.7); [exe](https://github.com/mikf/gallery-dl/releases/download/v1.27.7/gallery-dl.exe)) +Version **1.28.3** ([release](https://github.com/mikf/gallery-dl/releases/tag/v1.28.3); [exe](https://github.com/mikf/gallery-dl/releases/download/v1.28.3/gallery-dl.exe)) **The following sites use gallery-dl:** - [Pinterest](#pinterest) @@ -225,10 +225,10 @@ Version **1.27.7** ([release](https://github.com/mikf/gallery-dl/releases/tag/v1 ### YT-DLP -Version **2024.11.18** ([release](https://github.com/yt-dlp/yt-dlp/releases/tag/2024.11.18)) +Version **2024.12.23** ([release](https://github.com/yt-dlp/yt-dlp/releases/tag/2024.12.23)) -- x64 version - [exe](https://github.com/yt-dlp/yt-dlp/releases/download/2024.11.18/yt-dlp.exe) -- x86 version - [exe](https://github.com/yt-dlp/yt-dlp/releases/download/2024.11.18/yt-dlp_x86.exe) +- x64 version - [exe](https://github.com/yt-dlp/yt-dlp/releases/download/2024.12.23/yt-dlp.exe) +- x86 version - [exe](https://github.com/yt-dlp/yt-dlp/releases/download/2024.12.23/yt-dlp_x86.exe) **The following sites use yt-dlp:** - [YouTube](#youtube) @@ -237,10 +237,7 @@ Version **2024.11.18** ([release](https://github.com/yt-dlp/yt-dlp/releases/tag/ ### OF-Scraper -Version **3.10.7** ([release](https://github.com/datawhores/OF-Scraper/releases/tag/3.10.7)) - +Version **3.12.9** ([release](https://github.com/datawhores/OF-Scraper/releases/tag/3.12.9)) **Don't download the `exe` file. Always download `zip`.** @@ -365,6 +362,8 @@ The Reddit parser can parse data without cookies, but you can add it if you like - `mp4decrypt path` - the path to the `mp4decrypt.exe` - `key-mode-default` - change the key-mode. Default: `cdrm`. **Change this value only if you know what you are doing.** - `keydb_api` - change the keydb_api. **Change this value only if you know what you are doing.** + - `Private key` - path to the DRM key file `private_key.pem` + - `Client ID` - path to the DRM key file `client_id.bin` ![OnlyFans settings](https://raw.githubusercontent.com/AAndyProgram/SCrawler/main/ProgramScreenshots/SettingsSiteOnlyFans.png) @@ -400,10 +399,14 @@ Other settings **(change these values only if you know what you are doing)** 1. Open OnlyFans site settings 1. Add the path to the **ofscraper.exe** 1. Add the path to the **mp4decrypt.exe** +1. Fill in the paths to your DRM keys (if you use them) ([how to get them](https://forum.videohelp.com/threads/408031-Dumping-Your-own-L3-CDM-with-Android-Studio)) +1. If you use DRM keys, set `key-mode-default` option to `manual` 1. Click OK *[OF-Scraper guide](https://of-scraper.gitbook.io/of-scraper/)* +**DRM keys**: [OF-Scraper reference](https://of-scraper.gitbook.io/of-scraper/cdm-options#manual), **[complete guide](https://forum.videohelp.com/threads/408031-Dumping-Your-own-L3-CDM-with-Android-Studio)** + ### OnlyFans FAQ > How to download videos? @@ -674,13 +677,18 @@ The `User ID` and `UserAgent` values must be obtained from the page `https://jus --> ## YouTube -- `Download user videos` -- `Download user shorts` -- `Download user playlists` -- `Download user community: images` -- `Download user community: videos` -- `Ignore community errors` - if true, community errors will not be added to the log. -- `Use cookies` - default value for new users. Use cookies to download the user. +- Authorization + - `Use cookies` - default value for new users. Use cookies to download the user. +- New user defaults + - `Download user videos` + - `Download user shorts` + - `Download user playlists` + - `Download user community: images` + - `Download user community: videos` +- Communities + - `YouTube API host` - YouTube API instance host (YouTube-operational-API). Example: `localhost/YouTube-operational-API`, `http://localhost/YouTube-operational-API` ([how to create](#youtube-operational-api)) + - `YouTube API key` - YouTube Data API v3 developer key + - `Ignore community errors` - if true, community errors will not be added to the log. See additional settings [here](https://github.com/AAndyProgram/SCrawler/wiki/YouTube-downloader#Settings). @@ -691,6 +699,32 @@ See additional settings [here](https://github.com/AAndyProgram/SCrawler/wiki/You - [**yt-dlp**](#YT-DLP) - [**ffmpeg**](#ffmpeg) +### YouTube operational API +To download communities, you need to install the `YouTube operational API`. +The complete guide is [here](https://github.com/Benjamin-Loison/YouTube-operational-API/blob/476ae90317150deb2e187eb2dd2b59afc5b384a6/README.md#install-your-own-instance-of-the-api). + +1. Download and run [WampServer 3](https://sourceforge.net/projects/wampserver/files/latest/download). To install WampServer you'll need `VC Redistributable Packages`. You can download them all from [here](https://wampserver.aviatechno.net/) ([x86](https://wampserver.aviatechno.net/files/vcpackages/all_vc_redist_x86.zip), [x86 x64](https://wampserver.aviatechno.net/files/vcpackages/all_vc_redist_x86_x64.zip)). Install each package 2010, 2012, 2013, 2015. +2. Open command prompt +3. Type `cd C:\wamp64\www\` +4. Clone this repository by using `git clone https://github.com/Benjamin-Loison/YouTube-operational-API`. To use this command you'll need `Git`. You can download it from [here](https://git-scm.com/downloads/win). +5. Close command prompt +6. Download [composer](https://github.com/composer/windows-setup/releases/latest) +7. Download [protoc](https://github.com/protocolbuffers/protobuf/releases/latest) +8. Extract `protoc` files to `c:\wamp64\www\YouTube-operational-API\protoc\` +9. Add `c:\wamp64\www\YouTube-operational-API\protoc\bin\` to system paths +10. Install `composer` with default options +11. Open command prompt +12. Type `c:\wamp64\www\YouTube-operational-API\` +13. Enter the following command + +```bash +for /f "usebackq tokens=*" %a in (`dir /S /B "c:/wamp64/www/YouTube-operational-API/protoc/include/google/protobuf/"`) do protoc --php_out=proto/php/ --proto_path="c:/wamp64/www/YouTube-operational-API/protoc/include/google/protobuf/" %a +``` + +14. Launch WampServer +15. Verify that your API instance is reachable by trying to access: http://localhost/YouTube-operational-API/ +16. Add this URL to SCrawler YouTube settings `YouTube API host` + ## Pinterest - `Concurrent downloads` - the number of concurrent downloads. - `Saved posts user` - personal profile username.