2024.2.25.0

Home.md

@@ -105,8 +105,12 @@ Context menu:
 - `Open user` (with folder image) - open user folder
 - `Open user` (with globe) - open user site
 - `Open post` - open post in browser
+- `Copy to...` - copy item to another location
+- `Move to...` - move item to another location
 - `Add to Favorite` - this post will be added to your favorite feed
+- `Add to Favorite (remove from current)` - this post will be added to your favorite feed and removed from the current feed/session
 - `Add to special feed...`- this post will be added to the feed(s) you select (a selection form will open)
+- `Add to special feed (remove from current)...`- this post will be added to the feed(s) you select (a selection form will open) and removed from the current feed/session
 - `Remove from Favorite` - this post will be removed from your favorite feed
 - `Remove from special feed...` - this post will be removed from the feed(s) you select (a selection form will open)
 - `Find user` - find user in the main window
@@ -119,11 +123,15 @@ Load sessions:
 - `Load current session`
 - `Load last session` - load last saved session data
 - `Select loading session` - load the data of the session you have chosen (the session selection form will open)
+- `Copy checked to...` - copy checked items to another location
+- `Move checked to...` - move checked items to another location
 - `Load Favorite` - load your Favorite feed
 - `Load special feed` - load one or more of your special feeds
 - `Add checked to Favorite` - add checked posts to your Favorite feed
+- `Add checked to Favorite (remove from current)` - add checked posts to your Favorite feed and remove them from the current feed/session
 - `Remove checked from Favorite` - remove checked posts from your Favorite feed
 - `Add checked to special feed...` - add checked posts to one or more of your feeds
+- `Add checked to special feed (remove from current)...` - add checked posts to one or more of your feeds and remove them from the current feed/session
 - `Remove checked from special feed...` - remove checked posts from one or more of your feeds
 - `Clear Favorite` - remove all posts from your Favorite feed (just clearing the feed; **files will not be deleted**)
 - `Clear special feed...` - remove all posts from one or more of your feeds (just clearing the feeds; **files will not be deleted**)
@@ -132,17 +140,27 @@ Load sessions:
 - `Delete daily feed (by date)` - delete session feeds by the date (the date selection form will open; **files will not be deleted**)
 - `Merge sessions` - merge multiple session feeds into one
 - `Clear session` - clear list of recent downloads
+- `Merge special feeds` - merge multiple special feeds into one
+- `Select all/none`
 
 Change view:
 - `Downloads` - show downloaded regular users
 - `Subscriptions` - show downloaded [subscriptions](#subscriptions)
 
+**Move/Copy media:**
+- `Destination` - The destination folder for the media you want to move/copy. *If checked, the profile path will be ignored when changing profiles.*
+- `Place videos in a separate video folder` - The videos will be placed in a separate folder (`Video`) at the destination.
+- `Replace user profile` - The user profile will be replaced with the selected one. This may be suitable for cases where you're deleting a user profile. Feeds' media are associated with the user's profile. If you delete a profile, all of that profile's media will disappear from all of your feeds. If you change the user profile, this won't happen! **Only works when moving media!**
+- `Create path profile` - Create a path profile if it doesn't exist. This means that if you checked `Replace user profile` and didn't select a replacement profile, a new `Path` profile (path plugin) will be created and added to SCrawler. **Only works when moving media!**
+- `Profile` - List of user profiles (without path plugin). **Only works when moving media!**
+- `Profile (path)` - List of user profiles (path plugin only). **Only works when moving media!**
+
+![Feed move copy media](https://raw.githubusercontent.com/AAndyProgram/SCrawler/main/ProgramScreenshots/FeedMoveCopyTo.png)
+
 Hot-keys:
 - `F5` - refresh
 - `Ctrl+G` - go to a specific page
 
-*The buttons are only displayed if the `Store session data` option is [enabled](https://github.com/AAndyProgram/SCrawler/wiki/Settings#feed)*
-
 ![Feed session context menu](https://raw.githubusercontent.com/AAndyProgram/SCrawler/main/ProgramScreenshots/FeedWindowSessionContext.png)
 
 Hot keys
@@ -210,7 +228,9 @@ This allows you to pause the automation for a while. The pause will remain in ef
 ![Pause context](https://raw.githubusercontent.com/AAndyProgram/SCrawler/main/ProgramScreenshots/MainWindowPause.png)
 
 ### View
-- `View` - users view modes.
+
+Users view modes.
+
 - The first block specifies **how** users will be displayed in the form.
   - `Large/Small images` - with icons.
   - `List` / `Details` - simple, without icons.
@@ -237,6 +257,8 @@ This allows you to pause the automation for a while. The pause will remain in ef
   - `Limit dates (not in range)` - filter users whose last download date is not in the selected date range
   - `Limit dates (in range)` - filter users whose last download date is in the selected date range
 
+**You can also change the view by (un)grouping by site using the `Show groups` and `Use user grouping` [options in the settings](https://github.com/AAndyProgram/SCrawler/wiki/Settings#basis).**
+
 ![Main window view context](https://raw.githubusercontent.com/AAndyProgram/SCrawler/main/ProgramScreenshots/MainWindowView.png)
 
 ### LOG
@@ -281,6 +303,8 @@ Double-click the user to open the content folder.
 
 ## Download separate video
 
+`Download` - `Standalone downloader`
+
 Read [here](https://github.com/AAndyProgram/SCrawler/wiki/YouTube-downloader)
 
 ![Separate video downloader window](https://raw.githubusercontent.com/AAndyProgram/SCrawler/main/ProgramScreenshots/AppYouTube.png)
@@ -291,6 +315,7 @@ Read [here](https://github.com/AAndyProgram/SCrawler/wiki/YouTube-downloader)
 - [`Channels`](https://github.com/AAndyProgram/SCrawler/wiki/Channels)
 - [`Feed`](#feed)
 - [`Downloader`](#download-separate-video)
+- [`Scheduler`](https://github.com/AAndyProgram/SCrawler/wiki/Settings#automation) - show scheduler
 - `Show/Hide` - show/hide SCrawler
 - `Close` - close the program
 - `Close (no script)` - close the program ignore the script (if you configured the script in the settings)
@@ -313,6 +338,9 @@ Paste the copied URL into the `User name` field. **Ignore if the username is set
 
 If you want to add a folder as a user, you need to use the following pattern: `D:\Data\`.
 
+**ATTENTION!**
+<br/>**ALWAYS PASTE THE USER PROFILE URL**
+
 ![Add user](https://raw.githubusercontent.com/AAndyProgram/SCrawler/main/ProgramScreenshots/CreateUserClear.png)
 
 - `User name` - **Paste the user profile URL here.**
@@ -510,5 +538,3 @@ The program implements two ways to add users to this list:
     1. Check the `SpecialPath` values to make sure they still point to the correct locations. If not, replace these values with the correct ones. To find all non-empty `SpecialPath` values, you can use a simple regular expression: `SpecialPath="[^"]+"`.
     1. Save this file and close.
 1. Open SCrawler.
-
-

Plugins.md

@@ -545,6 +545,10 @@ Assembly attribute. Not used yet, but in future versions will allow SCrawler to
 - `Name` - your GitHub username
 - `RepoName` - your repository name
 
+## ReplaceInternalPluginAttribute
+
+If you want your plugin to overwrite the internal one, use this attribute.
+
 # PropertyValue value types
 
 | vb.net | C# | NET |

Settings.md

@@ -126,6 +126,7 @@
 - `Open last mode (users or subscriptions)` - if disabled, the user mode will be used when initializing the feed.
 - `Show friendly names instead of usernames`
 - `Show special feeds in media items` - if you have one or more special feeds, they will appear in the additional post context menu. Disabled by default. **Enabling this setting may increase memory consumption.**
+- `Update file location when moved` - the file location will be updated in the session data and in the feeds data.
 
 ![Feed settings](https://raw.githubusercontent.com/AAndyProgram/SCrawler/main/ProgramScreenshots/SettingsGlobalFeed.png)
 
@@ -172,24 +173,47 @@ Red highlight means that the field is required or you have entered an incorrect
 - `Get user media only` - the default for creating new users
 - `Temporary`, `Download images/videos` - personal site settings by default for creating new users. Means the same as the parameters of the same name on the `Defaults` tab. If the checkbox is in an intermediate state, then the same name setting of the `Defaults` tab will be used instead. Otherwise, this setting will be applied.
 
-## ffmpeg
+## Environment
+
+### ffmpeg
 
 ffmpeg is required for several sites like Reddit, PornHub, XHamster, XVIDEOS, etc. **It is included in the SCrawler release, but you can download ffmpeg from:**
 
 - x64 version - [release](https://github.com/GyanD/codexffmpeg/releases/tag/5.1.2); [zip](https://github.com/GyanD/codexffmpeg/releases/download/5.1.2/ffmpeg-5.1.2-full_build.zip); **version `5.1.2-full_build-www.gyan.dev`**
 - x86 version - [release](https://github.com/yt-dlp/FFmpeg-Builds/releases/tag/autobuild-2022-11-30-12-57); [zip](https://github.com/yt-dlp/FFmpeg-Builds/releases/download/autobuild-2022-11-30-12-57/ffmpeg-N-109274-gd7a5f068c2-win32-gpl.zip); **version `N-109457-geeb280f351-20221226`**
 
-## Gallery-dl
+### Gallery-dl
 
 Version **1.26.4** ([release](https://github.com/mikf/gallery-dl/releases/tag/v1.26.4); [exe](https://github.com/mikf/gallery-dl/releases/download/v1.26.4/gallery-dl.exe))
 
-## YT-DLP
+**The following sites use gallery-dl:**
+- [Pinterest](#pinterest)
+- [Twitter](#twitter)
+
+### YT-DLP
 
 Version **2023.11.16** ([release](https://github.com/yt-dlp/yt-dlp/releases/tag/2023.11.16))
 
 - x64 version - [exe](https://github.com/yt-dlp/yt-dlp/releases/download/2023.11.16/yt-dlp.exe)
 - x86 version - [exe](https://github.com/yt-dlp/yt-dlp/releases/download/2023.11.16/yt-dlp_x86.exe)
 
+**The following sites use yt-dlp:**
+- [YouTube](#youtube)
+- [ThisVid](#thisvid)
+- [TikTok](#tiktok) (using [plugin](https://github.com/bashonly/yt-dlp-TTUser))
+
+### OF-Scraper
+
+Version **2023.12.25_485586_main_d584b2b_package** ([release](https://github.com/datawhores/OF-Scraper/releases/tag/d584b2b))
+
+**Don't download the `exe` file. Always download `zip`.**
+
+**Don't put the program in the SCrawler's `Environment` folder because if you use the updater to update SCrawer, you'll lose 'OF-Scraper'. I recommend creating an `Environment2` folder and placing 'OF-Scraper' there.**
+
+**Don't rush to update 'OF-Scraper'. Users using 'OF-Scraper' report that minor releases contain many bugs.**
+
+*Go to the [OnlyFans OF-Scraper chapter](#of-scraper-support).*
+
 ## User query options
 
 **Available for search queries only. Not used when creating users.**
@@ -299,6 +323,8 @@ The Reddit parser can parse data without cookies, but you can add it if you like
 
 **To download DRM protected videos, you need to download [OF-Scraper](https://github.com/datawhores/OF-Scraper) and add it to the site settings in SCrawler.**
 
+**Don't forget to read [THIS](#of-scraper)!**
+
 1. Go to the [latest release](https://github.com/datawhores/OF-Scraper/releases/latest)
 1. Download zip file for Windows
 1. Extract it to any folder you want
@@ -338,11 +364,12 @@ To support the downloading of this site you should add the Mastodon domain to th
 
 ## Instagram
 - Authorization
-  - `x-csrftoken` - [how to find](#how-to-find-headers)
+  - `x-csrftoken` - [how to find](#how-to-find-headers) *(can be automatically extracted from cookies)*
   - `x-ig-app-id`, `ix-ig-www-claim`, `sec-ch-ua`, `sec-ch-ua-full-version-list`, `sec-ch-ua-platform-version`, `UserAgent` - [how to find](#how-to-find-headers)
   - `Saved posts user` - your personal Instagram username to download your saved posts
 - Other parameters
   - `Download timeline` - Download timeline *(with this setting, you can simply enable/disable the downloading of some Instagram blocks)*
+  - `Download reels` - Download reels *(with this setting, you can simply enable/disable the downloading of some Instagram blocks)*
   - `Download stories` - Download stories *(with this setting, you can simply enable/disable the downloading of some Instagram blocks)*
   - `Download stories: user` - Download active (non-pinned) user stories *(with this setting, you can simply enable/disable the downloading of some Instagram blocks)*
   - `Download tagged` - Download tagged posts *(with this setting, you can simply enable/disable the downloading of some Instagram blocks)*
@@ -350,6 +377,7 @@ To support the downloading of this site you should add the Mastodon domain to th
   - `Request time counter` - how many requests will be sent to Instagram before the program waits `Request timer` milliseconds (**it is highly recommended not to change this default value**)
   - `Post limit timer` - this is the time value (in milliseconds) the program will wait before processing the next request after 195 requests (**it is highly recommended not to change this default value**)
 - `Get timeline` - default value for new users
+- `Get reels` - default value for new users
 - `Get stories` - default value for new users
 - `Get stories: user` - default value for new users
 - `Get tagged photos` - default value for new users
@@ -359,6 +387,15 @@ To support the downloading of this site you should add the Mastodon domain to th
 
 ![Instagram settings](https://raw.githubusercontent.com/AAndyProgram/SCrawler/main/ProgramScreenshots/SettingsSiteInstagram.png)
 
+### Instagram data means
+1. Avatar *(download only for the first time)*
+2. `Reels`
+3. `Tagged posts`
+4. `Stories`
+5. `Stories: user`
+
+![Instagram site blocks](https://raw.githubusercontent.com/AAndyProgram/SCrawler/main/ProgramScreenshots/SettingsSiteInstagramData.png)
+
 ### Instagram requirements
 - **[COOKIES](#how-to-set-up-cookies) ARE REQUIRED ANYWAY**
 - [`x-csrftoken`](#how-to-find-headers)
@@ -459,6 +496,8 @@ result_cancel-->[*]
 
 The `Accept` and `UserAgent` headers must be obtained from the page `https://facebook.com/<USERNAME>`.
 
+If you can't find `x-ig-app-id` or `x-asbd-id`, you can get them from Instagram.
+
 ![Facebook settings](https://raw.githubusercontent.com/AAndyProgram/SCrawler/main/ProgramScreenshots/SettingsSiteFacebook.png)
 
 ### Facebook requirements
@@ -506,6 +545,7 @@ The `User ID` and `UserAgent` values must be obtained from the page `https://jus
 - `Download user playlists`
 - `Download user community: images`
 - `Download user community: videos`
+- `Ignore community errors` - if true, community errors will not be added to the log.
 - `Use cookies` - default value for new users. Use cookies to download the user.
 
 See additional settings [here](https://github.com/AAndyProgram/SCrawler/wiki/YouTube-downloader#Settings).
@@ -532,6 +572,8 @@ See additional settings [here](https://github.com/AAndyProgram/SCrawler/wiki/You
 - `Use native title` - use a user-created video title for the filename instead of the video ID
 - `Use native title in standalone downloader` - use a user-created video title for the filename instead of the video ID
 - `Add video ID to video title` - the video ID will be added to the file name
+- `Use regex to clean video title`
+- `Title regex` - regex to clean video title (only works with checked `Use regex to clean video title`)
 - `Use video date as file date` - set the file date to the date the video was added (website) (if available)
 
 ![TikTok settings](https://raw.githubusercontent.com/AAndyProgram/SCrawler/main/ProgramScreenshots/SettingsSiteTikTok.png)
@@ -725,6 +767,8 @@ This command can be a batch command or any script file (bat, ps1 or whatever you
 1. Scroll down to `Request Headers`.
 1. Find the headers you need, copy and paste them into the corresponding fields.
 
+**You can also use `Ctrl+F` to find the header you need**
+
 # How to find Imgur client ID
 1. Open browser developer tools
 1. Go to `Network`