diff --git a/Home.md b/Home.md
index 4d86d0e..5aaf4dd 100644
--- a/Home.md
+++ b/Home.md
@@ -105,8 +105,12 @@ Context menu:
- `Open user` (with folder image) - open user folder
- `Open user` (with globe) - open user site
- `Open post` - open post in browser
+- `Copy to...` - copy item to another location
+- `Move to...` - move item to another location
- `Add to Favorite` - this post will be added to your favorite feed
+- `Add to Favorite (remove from current)` - this post will be added to your favorite feed and removed from the current feed/session
- `Add to special feed...`- this post will be added to the feed(s) you select (a selection form will open)
+- `Add to special feed (remove from current)...`- this post will be added to the feed(s) you select (a selection form will open) and removed from the current feed/session
- `Remove from Favorite` - this post will be removed from your favorite feed
- `Remove from special feed...` - this post will be removed from the feed(s) you select (a selection form will open)
- `Find user` - find user in the main window
@@ -119,11 +123,15 @@ Load sessions:
- `Load current session`
- `Load last session` - load last saved session data
- `Select loading session` - load the data of the session you have chosen (the session selection form will open)
+- `Copy checked to...` - copy checked items to another location
+- `Move checked to...` - move checked items to another location
- `Load Favorite` - load your Favorite feed
- `Load special feed` - load one or more of your special feeds
- `Add checked to Favorite` - add checked posts to your Favorite feed
+- `Add checked to Favorite (remove from current)` - add checked posts to your Favorite feed and remove them from the current feed/session
- `Remove checked from Favorite` - remove checked posts from your Favorite feed
- `Add checked to special feed...` - add checked posts to one or more of your feeds
+- `Add checked to special feed (remove from current)...` - add checked posts to one or more of your feeds and remove them from the current feed/session
- `Remove checked from special feed...` - remove checked posts from one or more of your feeds
- `Clear Favorite` - remove all posts from your Favorite feed (just clearing the feed; **files will not be deleted**)
- `Clear special feed...` - remove all posts from one or more of your feeds (just clearing the feeds; **files will not be deleted**)
@@ -132,17 +140,27 @@ Load sessions:
- `Delete daily feed (by date)` - delete session feeds by the date (the date selection form will open; **files will not be deleted**)
- `Merge sessions` - merge multiple session feeds into one
- `Clear session` - clear list of recent downloads
+- `Merge special feeds` - merge multiple special feeds into one
+- `Select all/none`
Change view:
- `Downloads` - show downloaded regular users
- `Subscriptions` - show downloaded [subscriptions](#subscriptions)
+**Move/Copy media:**
+- `Destination` - The destination folder for the media you want to move/copy. *If checked, the profile path will be ignored when changing profiles.*
+- `Place videos in a separate video folder` - The videos will be placed in a separate folder (`Video`) at the destination.
+- `Replace user profile` - The user profile will be replaced with the selected one. This may be suitable for cases where you're deleting a user profile. Feeds' media are associated with the user's profile. If you delete a profile, all of that profile's media will disappear from all of your feeds. If you change the user profile, this won't happen! **Only works when moving media!**
+- `Create path profile` - Create a path profile if it doesn't exist. This means that if you checked `Replace user profile` and didn't select a replacement profile, a new `Path` profile (path plugin) will be created and added to SCrawler. **Only works when moving media!**
+- `Profile` - List of user profiles (without path plugin). **Only works when moving media!**
+- `Profile (path)` - List of user profiles (path plugin only). **Only works when moving media!**
+
+
+
Hot-keys:
- `F5` - refresh
- `Ctrl+G` - go to a specific page
-*The buttons are only displayed if the `Store session data` option is [enabled](https://github.com/AAndyProgram/SCrawler/wiki/Settings#feed)*
-
Hot keys
@@ -210,32 +228,36 @@ This allows you to pause the automation for a while. The pause will remain in ef
### View
-- `View` - users view modes.
- - The first block specifies **how** users will be displayed in the form.
- - `Large/Small images` - with icons.
- - `List` / `Details` - simple, without icons.
- - The second block specifies what **types of user** will be displayed in the form.
- - `Show users` - show regular users in user list.
- - `Show subscriptions` - show [subscriptions](#subscriptions) in user list.
- - The third block specifies the user **sites** that will be displayed in the form.
- - `All` - default
- - `Specific sites` - select sites to be displayed
- - The fourth block specifies **which** users will be displayed in the form.
- - `All` - default.
- - `Regular` - only show users who are not marked as `Favorite` or `Temporary`.
- - `Temporary` - only show users marked as temporary.
- - `Favorite` - only show users marked as favorite.
- - `Deleted` - only show users whose profiles have been deleted
- - `Suspended` - only show users whose profiles have been suspended
- - `Labels` - show users labeled with one or more selected label
- - `No Labels` - only show users without a label
- - `Excluded labels` - exclude from displaying users with the selected labels (this is suitable for situations where you want to hide a user but don't want to delete)
- - `Ignore excluded labels` - Ignore excluding labels. This allows you to show users that should be excluded from display without resetting the current selection of excluded labels.
- - `Show groups instead of labels` - show sites groups instead of labels (when filtering users by labels)
- - `Advanced filter` - advanced filter of users you want to display
- - The fifth block specifies the **date range** of the last download, whose users will be displayed in the form.
- - `Limit dates (not in range)` - filter users whose last download date is not in the selected date range
- - `Limit dates (in range)` - filter users whose last download date is in the selected date range
+
+Users view modes.
+
+- The first block specifies **how** users will be displayed in the form.
+ - `Large/Small images` - with icons.
+ - `List` / `Details` - simple, without icons.
+- The second block specifies what **types of user** will be displayed in the form.
+ - `Show users` - show regular users in user list.
+ - `Show subscriptions` - show [subscriptions](#subscriptions) in user list.
+- The third block specifies the user **sites** that will be displayed in the form.
+ - `All` - default
+ - `Specific sites` - select sites to be displayed
+- The fourth block specifies **which** users will be displayed in the form.
+ - `All` - default.
+ - `Regular` - only show users who are not marked as `Favorite` or `Temporary`.
+ - `Temporary` - only show users marked as temporary.
+ - `Favorite` - only show users marked as favorite.
+ - `Deleted` - only show users whose profiles have been deleted
+ - `Suspended` - only show users whose profiles have been suspended
+ - `Labels` - show users labeled with one or more selected label
+ - `No Labels` - only show users without a label
+ - `Excluded labels` - exclude from displaying users with the selected labels (this is suitable for situations where you want to hide a user but don't want to delete)
+ - `Ignore excluded labels` - Ignore excluding labels. This allows you to show users that should be excluded from display without resetting the current selection of excluded labels.
+ - `Show groups instead of labels` - show sites groups instead of labels (when filtering users by labels)
+ - `Advanced filter` - advanced filter of users you want to display
+- The fifth block specifies the **date range** of the last download, whose users will be displayed in the form.
+ - `Limit dates (not in range)` - filter users whose last download date is not in the selected date range
+ - `Limit dates (in range)` - filter users whose last download date is in the selected date range
+
+**You can also change the view by (un)grouping by site using the `Show groups` and `Use user grouping` [options in the settings](https://github.com/AAndyProgram/SCrawler/wiki/Settings#basis).**
@@ -281,6 +303,8 @@ Double-click the user to open the content folder.
## Download separate video
+`Download` - `Standalone downloader`
+
Read [here](https://github.com/AAndyProgram/SCrawler/wiki/YouTube-downloader)
@@ -291,6 +315,7 @@ Read [here](https://github.com/AAndyProgram/SCrawler/wiki/YouTube-downloader)
- [`Channels`](https://github.com/AAndyProgram/SCrawler/wiki/Channels)
- [`Feed`](#feed)
- [`Downloader`](#download-separate-video)
+- [`Scheduler`](https://github.com/AAndyProgram/SCrawler/wiki/Settings#automation) - show scheduler
- `Show/Hide` - show/hide SCrawler
- `Close` - close the program
- `Close (no script)` - close the program ignore the script (if you configured the script in the settings)
@@ -313,6 +338,9 @@ Paste the copied URL into the `User name` field. **Ignore if the username is set
If you want to add a folder as a user, you need to use the following pattern: `D:\Data\`.
+**ATTENTION!**
+
**ALWAYS PASTE THE USER PROFILE URL**
+
- `User name` - **Paste the user profile URL here.**
@@ -509,6 +537,4 @@ The program implements two ways to add users to this list:
1. Open the `Users.xml` file (in the SCrawler settings folder).
1. Check the `SpecialPath` values to make sure they still point to the correct locations. If not, replace these values with the correct ones. To find all non-empty `SpecialPath` values, you can use a simple regular expression: `SpecialPath="[^"]+"`.
1. Save this file and close.
-1. Open SCrawler.
-
-
+1. Open SCrawler.
\ No newline at end of file
diff --git a/Plugins.md b/Plugins.md
index 4657734..7e90f26 100644
--- a/Plugins.md
+++ b/Plugins.md
@@ -545,6 +545,10 @@ Assembly attribute. Not used yet, but in future versions will allow SCrawler to
- `Name` - your GitHub username
- `RepoName` - your repository name
+## ReplaceInternalPluginAttribute
+
+If you want your plugin to overwrite the internal one, use this attribute.
+
# PropertyValue value types
| vb.net | C# | NET |
diff --git a/Settings.md b/Settings.md
index 14d7a24..f6b584a 100644
--- a/Settings.md
+++ b/Settings.md
@@ -126,6 +126,7 @@
- `Open last mode (users or subscriptions)` - if disabled, the user mode will be used when initializing the feed.
- `Show friendly names instead of usernames`
- `Show special feeds in media items` - if you have one or more special feeds, they will appear in the additional post context menu. Disabled by default. **Enabling this setting may increase memory consumption.**
+- `Update file location when moved` - the file location will be updated in the session data and in the feeds data.
@@ -172,24 +173,47 @@ Red highlight means that the field is required or you have entered an incorrect
- `Get user media only` - the default for creating new users
- `Temporary`, `Download images/videos` - personal site settings by default for creating new users. Means the same as the parameters of the same name on the `Defaults` tab. If the checkbox is in an intermediate state, then the same name setting of the `Defaults` tab will be used instead. Otherwise, this setting will be applied.
-## ffmpeg
+## Environment
+
+### ffmpeg
ffmpeg is required for several sites like Reddit, PornHub, XHamster, XVIDEOS, etc. **It is included in the SCrawler release, but you can download ffmpeg from:**
- x64 version - [release](https://github.com/GyanD/codexffmpeg/releases/tag/5.1.2); [zip](https://github.com/GyanD/codexffmpeg/releases/download/5.1.2/ffmpeg-5.1.2-full_build.zip); **version `5.1.2-full_build-www.gyan.dev`**
- x86 version - [release](https://github.com/yt-dlp/FFmpeg-Builds/releases/tag/autobuild-2022-11-30-12-57); [zip](https://github.com/yt-dlp/FFmpeg-Builds/releases/download/autobuild-2022-11-30-12-57/ffmpeg-N-109274-gd7a5f068c2-win32-gpl.zip); **version `N-109457-geeb280f351-20221226`**
-## Gallery-dl
+### Gallery-dl
Version **1.26.4** ([release](https://github.com/mikf/gallery-dl/releases/tag/v1.26.4); [exe](https://github.com/mikf/gallery-dl/releases/download/v1.26.4/gallery-dl.exe))
-## YT-DLP
+**The following sites use gallery-dl:**
+- [Pinterest](#pinterest)
+- [Twitter](#twitter)
+
+### YT-DLP
Version **2023.11.16** ([release](https://github.com/yt-dlp/yt-dlp/releases/tag/2023.11.16))
- x64 version - [exe](https://github.com/yt-dlp/yt-dlp/releases/download/2023.11.16/yt-dlp.exe)
- x86 version - [exe](https://github.com/yt-dlp/yt-dlp/releases/download/2023.11.16/yt-dlp_x86.exe)
+**The following sites use yt-dlp:**
+- [YouTube](#youtube)
+- [ThisVid](#thisvid)
+- [TikTok](#tiktok) (using [plugin](https://github.com/bashonly/yt-dlp-TTUser))
+
+### OF-Scraper
+
+Version **2023.12.25_485586_main_d584b2b_package** ([release](https://github.com/datawhores/OF-Scraper/releases/tag/d584b2b))
+
+**Don't download the `exe` file. Always download `zip`.**
+
+**Don't put the program in the SCrawler's `Environment` folder because if you use the updater to update SCrawer, you'll lose 'OF-Scraper'. I recommend creating an `Environment2` folder and placing 'OF-Scraper' there.**
+
+**Don't rush to update 'OF-Scraper'. Users using 'OF-Scraper' report that minor releases contain many bugs.**
+
+*Go to the [OnlyFans OF-Scraper chapter](#of-scraper-support).*
+
## User query options
**Available for search queries only. Not used when creating users.**
@@ -299,6 +323,8 @@ The Reddit parser can parse data without cookies, but you can add it if you like
**To download DRM protected videos, you need to download [OF-Scraper](https://github.com/datawhores/OF-Scraper) and add it to the site settings in SCrawler.**
+**Don't forget to read [THIS](#of-scraper)!**
+
1. Go to the [latest release](https://github.com/datawhores/OF-Scraper/releases/latest)
1. Download zip file for Windows
1. Extract it to any folder you want
@@ -338,11 +364,12 @@ To support the downloading of this site you should add the Mastodon domain to th
## Instagram
- Authorization
- - `x-csrftoken` - [how to find](#how-to-find-headers)
+ - `x-csrftoken` - [how to find](#how-to-find-headers) *(can be automatically extracted from cookies)*
- `x-ig-app-id`, `ix-ig-www-claim`, `sec-ch-ua`, `sec-ch-ua-full-version-list`, `sec-ch-ua-platform-version`, `UserAgent` - [how to find](#how-to-find-headers)
- `Saved posts user` - your personal Instagram username to download your saved posts
- Other parameters
- `Download timeline` - Download timeline *(with this setting, you can simply enable/disable the downloading of some Instagram blocks)*
+ - `Download reels` - Download reels *(with this setting, you can simply enable/disable the downloading of some Instagram blocks)*
- `Download stories` - Download stories *(with this setting, you can simply enable/disable the downloading of some Instagram blocks)*
- `Download stories: user` - Download active (non-pinned) user stories *(with this setting, you can simply enable/disable the downloading of some Instagram blocks)*
- `Download tagged` - Download tagged posts *(with this setting, you can simply enable/disable the downloading of some Instagram blocks)*
@@ -350,6 +377,7 @@ To support the downloading of this site you should add the Mastodon domain to th
- `Request time counter` - how many requests will be sent to Instagram before the program waits `Request timer` milliseconds (**it is highly recommended not to change this default value**)
- `Post limit timer` - this is the time value (in milliseconds) the program will wait before processing the next request after 195 requests (**it is highly recommended not to change this default value**)
- `Get timeline` - default value for new users
+- `Get reels` - default value for new users
- `Get stories` - default value for new users
- `Get stories: user` - default value for new users
- `Get tagged photos` - default value for new users
@@ -359,6 +387,15 @@ To support the downloading of this site you should add the Mastodon domain to th
+### Instagram data means
+1. Avatar *(download only for the first time)*
+2. `Reels`
+3. `Tagged posts`
+4. `Stories`
+5. `Stories: user`
+
+
+
### Instagram requirements
- **[COOKIES](#how-to-set-up-cookies) ARE REQUIRED ANYWAY**
- [`x-csrftoken`](#how-to-find-headers)
@@ -459,6 +496,8 @@ result_cancel-->[*]
The `Accept` and `UserAgent` headers must be obtained from the page `https://facebook.com/`.
+If you can't find `x-ig-app-id` or `x-asbd-id`, you can get them from Instagram.
+
### Facebook requirements
@@ -506,6 +545,7 @@ The `User ID` and `UserAgent` values must be obtained from the page `https://jus
- `Download user playlists`
- `Download user community: images`
- `Download user community: videos`
+- `Ignore community errors` - if true, community errors will not be added to the log.
- `Use cookies` - default value for new users. Use cookies to download the user.
See additional settings [here](https://github.com/AAndyProgram/SCrawler/wiki/YouTube-downloader#Settings).
@@ -532,6 +572,8 @@ See additional settings [here](https://github.com/AAndyProgram/SCrawler/wiki/You
- `Use native title` - use a user-created video title for the filename instead of the video ID
- `Use native title in standalone downloader` - use a user-created video title for the filename instead of the video ID
- `Add video ID to video title` - the video ID will be added to the file name
+- `Use regex to clean video title`
+- `Title regex` - regex to clean video title (only works with checked `Use regex to clean video title`)
- `Use video date as file date` - set the file date to the date the video was added (website) (if available)
@@ -725,6 +767,8 @@ This command can be a batch command or any script file (bat, ps1 or whatever you
1. Scroll down to `Request Headers`.
1. Find the headers you need, copy and paste them into the corresponding fields.
+**You can also use `Ctrl+F` to find the header you need**
+
# How to find Imgur client ID
1. Open browser developer tools
1. Go to `Network`