2025.7.18.0

API.Instagram: fix special folder issue
API.OnlyFans: bypass unpurchased videos; add support for GIF files
API.Reddit: add OAuth credentials validation; add extended 429 error handling
API.Xhamster: remove 'UserOptions' function ('SiteSettings'); add support for downloading 'moments'
API.XVIDEOS: remove 'UserOptions' function ('SiteSettings'); remove 'UserExchangeOptions' class
Add 'EditorExchangeOptionsBase_P' and update base classes for user options

.gitattributes

@@ -1,2 +1,5 @@
 # Auto detect text files and perform LF normalization
 * text=auto
+
+# Declare files that will always have CRLF line endings on checkout.
+*.md text eol=crlf

.github/FUNDING.yml

@@ -0,0 +1,13 @@
+# These are supported funding model platforms
+
+github: # Replace with up to 4 GitHub Sponsors-enabled usernames e.g., [user1, user2]
+patreon: # Replace with a single Patreon username
+open_collective: # Replace with a single Open Collective username
+ko_fi: #andyprogram
+tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
+community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
+liberapay: # Replace with a single Liberapay username
+issuehunt: # Replace with a single IssueHunt username
+otechie: # Replace with a single Otechie username
+lfx_crowdfunding: # Replace with a single LFX Crowdfunding project-name e.g., cloud-foundry
+custom: ['https://github.com/AAndyProgram/SCrawler/blob/main/HowToSupport.md']

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,46 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: "[BUG]"
+labels: bug
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Steps to reproduce the behavior:
+1. **Profile URL**: 
+2. **Post URL**: 
+3. Do something
+4. See error
+
+<details>
+<summary>Log data</summary>
+<pre>
+If the program log contains any data, replace this line with the log data. If the program log does not contain any data, write here about.
+</pre>
+</details>
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem.
+
+**Release information:**
+
+**Please complete the following information or replace the following text with data copied from SCrawler (click the top right info button in the main window, then the `Environment` button, then the `Copy` button, and paste the copied text here).**
+ - OS: [e.g. Windows 10, Windows 11]
+ - Architecture: [e.g. x86, x64]
+ - Version: [e.g. 2023.3.5.0]
+ - NET.Framework version: 
+ - ffmpeg version (command `ffmpeg -version`):
+ - yt-dlp version (command `yt-dlp --version`):
+ - gallery-dl version (command `gallery-dl --version`):
+ - cURL version (command `curl --version`):
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/custom.md

@@ -0,0 +1,10 @@
+---
+name: Custom issue template
+about: Describe this issue template's purpose here.
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,20 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: "[REQUEST]"
+labels: enhancement
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

.github/ISSUE_TEMPLATE/plugin_add.md

@@ -0,0 +1,12 @@
+---
+name: I developed a plugin for SCrawler
+about: I developed a plugin for SCrawler. Add plugin to plugin list.
+title: "[NEW PLUGIN]"
+labels: 'New Plugin'
+assignees: ''
+
+---
+
+Plugin address:
+Plugin name:
+Plugin site:

.gitignore

@@ -9,8 +9,8 @@
 *.user
 *.userosscache
 *.sln.docstates
-ffmpeg.exe
-ToDo.txt
+.obsidian/
+BugReporterFormDiscordWebHook.vb
 
 # User-specific files (MonoDevelop/Xamarin Studio)
 *.userprefs
@@ -33,7 +33,6 @@ bld/
 [Oo]bj/
 [Ll]og/
 [Ll]ogs/
-ffmpeg/
 Info/
 
 # Visual Studio 2015/2017 cache/options directory

CONTRIBUTING.md

@@ -1,42 +1,45 @@
 # Contributor's Guide
 
-I welcome requests! Follow these steps to contribute:
-
+Follow these steps to contribute:
 1. Find an [issue](https://github.com/AAndyProgram/SCrawler/issues) that needs assistance.
-2. Let me know you are working on it by posting a comment on the issue.
-3. If you find an error in the code, please provide a link to the file and the line number.
-4. If you have a suggestion to change the code, you can post a block of code to replace. I don't currently have time to learn pull requests, so it might work this way.
+1. Let me know you're working on this by posting a comment on this issue.
+1. If you find a bug in the code, please provide a link to the file and line number.
+1. If you have a code change suggestion, you can post a replacement code block.<!-- I also accept pull requests.-->
+
+# How to report a problem
+
+**[Read here](https://github.com/AAndyProgram/SCrawler/blob/main/FAQ.md#how-to-report-a-problem)**
 
 # How to build from source
+1. Delete the `PersonalUtilities` project from the solution.
+1. Delete the `PersonalUtilities.Notifications` project from the solution.
+1. The following libraries must be added to project references with the '**Copy to output folder**' option:
+    - `PersonalUtilities.dll`
+    - `PersonalUtilities.Notifications.dll`
+    - `Microsoft.Toolkit.Uwp.Notifications.dll`
+    - `System.ValueTuple.dll`
+1. Import `PersonalUtilities.Functions` for the whole project.
 
-1. Delete the "PersonalUtilities" project from the solution.
-2. Add the latest version of the "PersonalUtilities.dll" library (from the [latest release](https://github.com/AAndyProgram/SCrawler/releases/latest)).
-3. Import PersonalUtilities.Functions for the whole project.
-
-**Always use the correct "PersonalUtilities.dll" library. You must download this library from the release of the code you downloaded.**
+**Always use the correct libraries. You must download libraries from the same release date as the code commit date.**
 
 # How to request a new site
 
+**I'm currently not accepting requests to develop new sites.**
+
 1. Check [issues](https://github.com/AAndyProgram/SCrawler/issues) (open and [closed](https://github.com/AAndyProgram/SCrawler/issues?q=is%3Aissue+is%3Aclosed)) and [discussions](https://github.com/AAndyProgram/SCrawler/discussions) to find your issue. Perhaps I have already answered your request.
-2. If you don't find anything, create a new issue with your request. I usually reply as soon as possible (within the next few hours).
-    - If I'm interested in a site you want to add, it may be added in future releases.
-      - If the site has an API that does not require authorization, it may be added in the coming releases.
-      - You can make it faster by posting a link to the API. **I don't use OAuth authentication** in my application, so if it's not too hard to make a new parsing algorithm **without OAuth** authorization, I can start developing it in the coming days. Otherwise, I need time to figure out how to do it.
-      - If the site does not have an API that does not require authorization, this may take some time.
-	- If you will be posting request urls **without OAuth** authentication, I might consider adding your site if I have time.
-    - If I'm **not** interested in the site you want to add, you can pay to have it added by making a donation of approximately $10. **But before that, you still need to create an issue. If I'm not interested, you can offer me a deal to develop it for money. I'll check the site you want to add, check the availability of the API and tell you how much time I need to develop it and the price. If you agree, I will do it.** [![ko-fi](https://www.ko-fi.com/img/githubbutton_sm.svg)](https://ko-fi.com/andyprogram)
-    
-    
+1. If you don't find anything, create a new issue with your request.
+
+# Requirements for new site requests
+
+**Attention! I'll add a new site only if I'm interested. I also have a life, and any development takes time.**
+
+- Post a link to the site's API
+- Post request URLs **without OAuth** authentication
+- Post a **complete cURL** request which provides the required information (JSON is better)
+
+**I don't use OAuth authentication** in my application, so if it's not too hard to make a new parsing algorithm **without OAuth** authorization, I can start developing it in the coming days. Otherwise, I need time to figure out how to do it.
+
+If I'm interested in a site you want to add, it may be added in future releases.
+
 # Sites I will never develop
-
-- Facebook
-
-# Sites requested by users
-
-- TikTok
-  - API for receiving data without authorization was not found. Therefore, I don't have time to start developing this site parsing algorithm. If anyone knows of requests that may collect data without OAuth authentication, please let me know.
-  
-# Contact me
-
-[Element messenger](https://element.io/): @andyprogram:matrix.org
-https://matrix.to/#/@andyprogram:matrix.org
+- Tumblr

FAQ.md

@@ -0,0 +1,100 @@
+
+**Join our Discord server**: https://discord.gg/uFNUXvFFmg
+<br/>*You can get help faster there!*
+
+# Docs
+- Basic info: https://github.com/AAndyProgram/SCrawler/blob/main/README.md
+- **GUIDE**: https://github.com/AAndyProgram/SCrawler/wiki/
+- Settings: https://github.com/AAndyProgram/SCrawler/wiki/Settings
+- Discord: https://discord.gg/uFNUXvFFmg
+
+Most of your questions are already answered. All settings, functions, buttons and everything else described in the guide.
+
+# Backup
+I strongly recommend you to **regularly** create backup copies of the settings files. **An [example script](https://github.com/AAndyProgram/SCrawler/blob/main/Tools/ArchiveSCrawlerUsersDataFiles.bat) for this** on GitHub (you **should adapt** it to your environment, and you can use it when [SCrawler is closed](https://github.com/AAndyProgram/SCrawler/wiki/Settings#behavior)).
+
+**This way you'll always have the latest backup of your settings files and can restore it if something goes wrong!**
+
+# How to report a problem
+1. **Post your problem [here](https://github.com/AAndyProgram/SCrawler/issues) or in the [help channel](https://discord.com/channels/1124032649682493462/1124281838056259614) on our Discord server**
+2. Attach the **profile URLs or links** that you cannot download.
+3. Attach the **LOG** if it exists.
+4. Attach **the environment information** copied from SCrawler (click the top right info button in the main window, then the `Environment` button, then the `Copy` button, and paste the copied text into the message).
+5. *Add screenshots to illustrate the problem (**optional**)*
+
+**ATTENTION! Issues without URLs will be closed without a response!**
+
+# Most frequently questions about SCrawler
+
+**If something doesn't download, always check the [SITE'S REQUIREMENTS](https://github.com/AAndyProgram/SCrawler/wiki/Settings#sites-requirements) before asking questions!**
+
+*How to use: find your problem in the list and read the answer.*
+
+## General questions
+- **PROFILES**
+    - I added a profile but **nothing downloaded** :arrow_forward: check your cookies and [site requirements](https://github.com/AAndyProgram/SCrawler/wiki/Settings#sites-requirements). If there are any optional fields that you don't fill in, do so. Still nothing works - [report it](#how-to-report-a-problem)!
+    - User downloading failed :arrow_forward: check your credentials and **[SITES REQUIREMENTS](https://github.com/AAndyProgram/SCrawler/wiki/Settings#sites-requirements)**. If all settings are set and nothing works, [report it](#how-to-report-a-problem). **Don't forget to attach the LOG.**
+    - [How to redownload user](https://github.com/AAndyProgram/SCrawler/wiki#redownload-user)
+    - How to **add profile** to download :arrow_forward: copy the **[profile URL](https://github.com/AAndyProgram/SCrawler/wiki#add-user)** and press `Insert` or `Ctrl+Insert`. **ALWAYS PASTE THE USER PROFILE URL**. After that select this user and press `F5` or click the `Download selected` button.
+- How to download **[saved posts](https://github.com/AAndyProgram/SCrawler/wiki#saved-posts)**
+- **[HOW TO ADD COOKIES](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-set-up-cookies)**
+- [How to report a problem](#how-to-report-a-problem)
+- I want you to **add the site** to SCrawler :arrow_forward: **I'm not currently accepting requests to add new sites**, but you can [create a plugin](https://github.com/AAndyProgram/SCrawler/wiki/Plugins) (for your site) for SCrawler.
+- What language is SCrawler written in :arrow_forward: `vb.net`
+- I don't know `vb.net` and I can't write a plugin :arrow_forward: you can write a plugin in `C#`
+- I have a suggestion, will it be added :arrow_forward: maybe if it interested me.
+- How to name files using a pattern (e.g. `Site_PostID_Name.jpg`) :arrow_forward: **there is no such functionality and there are no such plans**.
+- **DON'T CHANGE THE DEFAULT SITE SETTINGS UNLESS YOU KNOW EXACTLY WHAT YOU'RE DOING!** SCrawler already has all the default settings to work. You only need to add credentials (where [required](https://github.com/AAndyProgram/SCrawler/wiki/Settings#sites-requirements)).
+- My computer shut down while SCrawler was running and now **SCrawler won't start or some users are missing** :arrow_forward: restore user settings from [backup](#backup).
+- Installation, update and configuration
+    - How to install: https://github.com/AAndyProgram/SCrawler#installation
+    - How to update: https://github.com/AAndyProgram/SCrawler#updating
+    - What file executes the program: **`SCrawler.exe`**
+    - Where to find binaries: https://github.com/AAndyProgram/SCrawler/releases/latest
+	- [How to build from source](https://github.com/AAndyProgram/SCrawler/blob/main/CONTRIBUTING.md#how-to-build-from-source)
+    - [Video how to configure](#video-how-to-configure)
+- **Antivirus**
+    - **Antivirus detects SCrawler as a virus** :arrow_forward: SCrawler doesn't contain any viruses at all. All code is posted on GitHub. You can review it. I have nothing to hide. SCrawler just downloads pictures and videos. That's all. If you trust SCrawler, you should just add it to the antivirus exceptions, as I did. Sometimes antiviruses identify SCawler as a virus. This is usually related to the number of files being edited (users' settings files) and the number of files being downloaded. In this case, the antivirus can also remove these files, which will damage users' settings. **If you don't trust SCrawler, just delete it.**
+    - **Antivirus detects gallery-dl as a virus** :arrow_forward: it's a trustworthy program that is trusted by thousands of people around the world. Antiviruses identify some builds as containing viruses, but this is not true. **If you don't trust gallery-dl, you can simply delete it. But if you delete it, you won't be able to download [Twitter & Pinterest](https://github.com/AAndyProgram/SCrawler/wiki/Settings#gallery-dl).** You should decide for yourself.
+
+## Sites questions
+
+*How to use: find the site you need in the list and read the answer.*
+
+- Reddit: don't use credentials at all or configure [OAuth](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-get-reddit-credentials). **Reddit profiles can be downloaded without any credentials at all. Subreddits require OAuth! If nothing downloads, use OAuth!** Don't use OAuth token to download saved posts (use cookies only).
+- **META** (**Instagram**, Threads, Facebook): you need **cookies** and fill in **all fields**
+- **Instagram [TIPS](https://github.com/AAndyProgram/SCrawler/wiki/Settings#instagram-tips)**
+- **Instagram saved posts**: I don't consider questions like "I have 10k saved posts and only 1000 were downloaded". Download posts, remove them from saved posts, delete the `Saved posts` **settings folder**, repeat.
+- TikTok: works via yt-dlp. If something doesn't download, we need to wait until yt-dlp fixes it. TikTok doesn't require cookies to download.
+- Porn sites: **COOKIES**!
+- ThisVid: https://github.com/AAndyProgram/SCrawler/wiki/Settings#thisvid-faq
+- **OnlyFans**: cookies + **all fields** + [OF-Scraper (download the correct version that I pointed)](https://github.com/AAndyProgram/SCrawler/wiki/Settings#of-scraper) & [mp4decrypt](https://www.bento4.com/downloads/) & **DRM keys** to download DRM protected videos. [OF-Scraper support](https://github.com/AAndyProgram/SCrawler/wiki/Settings#of-scraper-support). Also read [this](https://github.com/AAndyProgram/SCrawler/wiki/Settings#onlyfans-faq)
+- **JustForFans**: **THE VIDEO ISN'T DOWNLOADING AT THE MOMENT** ([Issue](https://discord.com/channels/1124032649682493462/1205547615199039551/1231349555132366870))
+
+## Other questions
+
+### Does the program remember the last download and check for new posts, downloading only new posts, or does the program download the entire profile every time
+The program stored posts IDs in users' folders. For the first time, the program downloads the entire profile. All subsequent times the program will check for new posts and download **only new posts**!
+
+### Does this program have a GUI or CLI, and will a CLI be added in the future
+This is a GUI program and **NO**, <u>CLI will not be added</u>
+
+### How to remove the label
+There is no functionality to remove an individual label. You can open the `Labels.txt` file in the program settings folder and delete any label you want. You also can delete this file (`Labels.txt`). In this case, when SCrawler is launched, the list of labels will be populated only with existing labels (from the user data files).
+
+### How to remove a user from the blacklist
+Just add that user back to the program. In the dialog box that opens, click the `Add and remove from blacklist` button.
+
+### You lost me. Your program is too complicated.
+**I'm fine with that**. If the program is too complicated for you or you can't configure it, I can only suggest you find another (easier) program. I really don't mind! The program is free. I develop SCrawler for myself and publish it on GitHub because people found my program useful. If someone can't use it or doesn't like it, I'm okay with it.
+
+### Add a step-by-step guide or video on how to use the program
+**NO!** The guide fully covers all the functionality of SCrawler! If you don't respect my work, I don't waste my time. If you want, you can create a video tutorial and send it to me. Then I'll add it. All options and their purposes are described on the wiki. The wiki also contains a description of all the settings and how to configure them. For complex settings there is a step-by-step guide. Read the [main](README.md) information and [GUIDE](https://github.com/AAndyProgram/SCrawler/wiki/) and you won't have any problems. I've developed a program with an intuitive interface. There is a `Settings` button, download buttons, a context menu that appears when you right-click on a user, and other controls. Anyone can use it.
+
+**There is already a [video](#video-how-to-configure) example of how to configure a site.**
+
+# Video how to configure
+
+**The following video was recorded by a user who loves SCrawler and demonstrates how to add credentials using Instagram as an example:**
+
+[![How to configure](https://img.youtube.com/vi/XDn7zG4I700/0.jpg)](https://www.youtube.com/watch?v=XDn7zG4I700)

HowToSupport.md

@@ -0,0 +1,12 @@
+Your support is very valuable to me. Any support is greatly appreciated. Your support encourages me to make new features, update the program, add new sites, etc.
+
+You can support the program by:
+  - **Bitcoin**: BC1Q0NH839FT5TA44DD7L7RLR97XDQAG9V8D6N7XET
+  - :repeat: make a post about my program on your profile (Reddit, Twitter, Instagram and any other social networks)
+  - :speech_balloon: tell your friends about the program
+  - :heart: like the program on this site: https://alternativeto.net/software/scrawler/about/
+  - :heart: like the program on this site: https://www.softpedia.com/get/Internet/Download-Managers/Social-networks-crawler.shtml
+  - suggest my program as an alternative ([on this site](https://alternativeto.net/software/scrawler/about/)) to any program you have used before
+
+I would be very grateful for any support! :blush:
+<!---:heavy_dollar_sign: make a donation on this site: https://ko-fi.com/andyprogram--->

Plugins.md

@@ -0,0 +1,10 @@
+You can create a plugin for any site you want. **To create a plugin, read [this guide](https://github.com/AAndyProgram/SCrawler/wiki/Plugins).**
+
+If you've created a plugin, you can create a [new issue](https://github.com/AAndyProgram/SCrawler/issues/new?assignees=&labels=New+Plugin&projects=&template=plugin_add.md&title=%5BNEW+PLUGIN%5D) and I'll add your plugin to the list below.
+
+----
+
+List of available plugins:
+
+Tools:
+- [image2post](https://github.com/unknown81311/SCrawler-image2post) by @unknown81311: **get reddit post URL from file.**

ProgramsComparison.md

@@ -0,0 +1,131 @@
+# yt-dlp
+
+https://github.com/yt-dlp/yt-dlp/
+
+**Great powerful CLI tool that supports hundreds of sites.**
+
+SCrawler has advanced user management, collections, labels, groups, automatic downloads, a beautiful view, GUI, the ability to add plugins for other sites and much more. Just try it and compare.
+
+# gallery-dl
+
+https://github.com/mikf/gallery-dl
+
+**Great powerful CLI tool that supports hundreds of sites.**
+
+SCrawler has advanced user management, collections, labels, groups, automatic downloads, a beautiful view, GUI, the ability to add plugins for other sites and much more. Just try it and compare.
+
+# 4K Video Downloader
+
+https://www.4kdownload.com/-plbrz/video-downloader
+
+| Option | SCrawler | 4K Video Downloader |
+| ---- | ---- | ---- |
+| User managament | **Advanced** | No |
+| Automatic downloads | **Yes** | No |
+| Downloading groups | **Yes** | No |
+| Labeling users | **Yes** | No |
+| Filtering | **Yes** | No |
+| Collections | **Yes** | No |
+| Specific user folders | **Yes** | No |
+| Favorite / Temporary user options | **Yes** | No |
+| Plugins support | **Yes** | No |
+| Download single video | **Unlimited** | 30 videos per day *(unlimited starts from 12 EUR)* |
+| Download videos per channel | **Unlimited** | 5 free *(unlimited starts from 12 EUR)* |
+| Download videos per playlist | **Unlimited** | 10 free *(unlimited starts from 12 EUR)* |
+| Download video subtitles | **Any for free**: single video, playlist, user/channel, album, etc| Free for single video |
+| The number of subtitles you can download for a video | **All of them** | Up to 10 |
+| Convert subtitles to additional formats | **Yes** | No |
+| Support LRC format | **Yes** | No |
+| Select audio codec for audio/video | **Yes** | No |
+| Extract and convert additional audio tracks for video | **Yes** | No |
+| Simultaneous downloads | **Unlimited** | 1 free, 3 for 12 EUR, 7 for 43 EUR|
+| Private YouTube content download | **Free** | Only in paid plans *starts from 12 EUR* |
+| **Paid** | **No** | Yes |
+| **Free options** | **The program is completely free** | Only **30** videos per day, 5 from a channel, 10 from a playlist |
+| Permitted Commercial Use | **Yes** | Starting from 43 EUR |
+| Automatic Subscriptions Update | **Free** | Paid (43 EUR) |
+| Posts and Captions Export | No | Paid (43 EUR) |
+| Advertisements free | **No ADs at all for free** | Paid (43 EUR) |
+| Operating Systems | Windows 10+ | Windows 7+, MacOS 10.13+, Ubuntu x64 |
+| Select want content type to download | **Yes** | No |
+| Instagram support | **Yes** | No |
+| Twitter support | **Yes** | No |
+| Reddit support | **Yes** | No |
+| Other sites support | **Yes** | No |
+| Still supported | Yes | Yes |
+
+# 4K Stogram
+
+<!---https://www.4kdownload.com/products/product-stogram--->
+https://www.4kdownload.com/-ad0p9/stogram
+
+| Option | SCrawler | 4K Stogram |
+| ---- | ---- | ---- |
+| User managament | **Advanced** | Primitive |
+| Automatic downloads | **Yes** | No |
+| Downloading groups | **Yes** | No |
+| Labeling users | **Yes** | No |
+| Filtering | **Yes** | No |
+| Collections | **Yes** | No |
+| Specific user folders | **Yes** | No |
+| Favorite / Temporary user options | **Yes** | No |
+| Plugins support | **Yes** | No |
+| Download posts by username | Yes | Yes |
+| Download posts by hashtag | No | **Yes** |
+| Download posts by location | No | **Yes** |
+| Save Private Instagram Content with Permission| Yes | Yes |
+| Download Instagram Stories and Highlights | Yes | Yes |
+| See Others Instagram Feed As Your Own | Yes | Yes |
+| Download Instagram Video Posts | Yes | Yes |
+| Backup Your Instagram Account | Yes | Yes |
+| Save Instagram Posts by Date | Yes | Yes |
+| Download Instagram Saved Posts | Yes | Yes |
+| Download Instagram Tagged Posts | Yes | Yes |
+| Export and import subscriptions | No | **Yes** |
+| **Paid** | **No** | Yes |
+| **Free options** | **The program is completely free** | Only **ONE** profile downloading and only **200 posts** per day |
+| Permitted Commercial Use | **Yes** | Starting from 43 EUR |
+| Automatic Subscriptions Update | **Free** | Paid (43 EUR) |
+| Posts and Captions Export | No | Paid (43 EUR) |
+| Advertisements free | **No ADs at all for free** | Paid (18 EUR) |
+| Operating Systems | Windows 10+ | Windows 7+, MacOS 10.13+, Ubuntu x64 |
+| Select want content type to download | **Yes** | No |
+| Instagram support | Yes | Yes |
+| Twitter support | **Yes** | No |
+| Reddit support | **Yes** | No |
+| Other sites support | **Yes** | No |
+| Still supported | Yes | Yes |
+
+# RipMeApp
+
+https://github.com/RipMeApp/ripme
+
+| Option | SCrawler | RipMeApp |
+| ---- | ---- | ---- |
+| User managament | **Advanced** | No |
+| Automatic downloads | **Yes** | No |
+| Downloading groups | **Yes** | No |
+| Labeling users | **Yes** | No |
+| Filtering | **Yes** | No |
+| Collections | **Yes** | No |
+| Specific user folders | **Yes** | No |
+| Favorite / Temporary user options | **Yes** | No |
+| Plugins support | **Yes** | No |
+| Download posts by username | Yes | Yes |
+| Download posts by hashtag | No | No |
+| Download posts by location | No | No |
+| Save Private Instagram Content with Permission| Yes | Yes |
+| Download Instagram Stories | Yes | Yes |
+| Download Instagram Video Posts | Yes | Yes |
+| Backup Your Instagram Account | Yes | Yes |
+| Download Instagram Saved Posts | **Yes** | No |
+| Download Instagram Tagged Posts | **Yes** | No |
+| Export and import subscriptions | No | No |
+| **Paid** | **No** | **No** |
+| **Free options** | The program is completely free | The program is completely free, but site limits are not declared |
+| Operating Systems | Windows 10+ | Windows, MacOS, Linux |
+| Select want content type to download | Yes | Yes |
+| Suported sites | 15+ internal and any site using plugins | 86+ sites (declared) |
+| Other sites support | **Yes** | No |
+| Still supported | **Yes** | **No (last release date May 4, 2021)** |
+

README.md

@@ -1,98 +1,221 @@
-# Social networks crawler
+# 🏳️‍🌈 Happy LGBT Pride Month 🎉
 
-A program to download photo and video from Reddit, Twitter, Instagram, [etc](#supported-sites).
+# 🏳️‍🌈 Social networks crawler 🏳️‍🌈
 
-Do you like this program? Consider adding to my coffee fund by making a donation to show your support. :)
+[![GitHub release (latest by date)](https://img.shields.io/github/v/release/AAndyProgram/SCrawler)](https://github.com/AAndyProgram/SCrawler/releases/latest)
+[![GitHub license](https://img.shields.io/github/license/AAndyProgram/SCrawler)](https://github.com/AAndyProgram/SCrawler/blob/main/LICENSE)
+[![Discord](https://img.shields.io/discord/1124032649682493462?logo=discord)](https://discord.gg/uFNUXvFFmg)
+[![GitHub all releases](https://img.shields.io/github/downloads/aandyprogram/scrawler/total?label=Total%20downloads)](https://github.com/AAndyProgram/SCrawler/releases)
+[![FAQ](https://img.shields.io/badge/FAQ-green)](FAQ.md)
+[![GUIDE](https://img.shields.io/badge/GUIDE-green)](https://github.com/AAndyProgram/SCrawler/wiki)
+[![How to support](https://img.shields.io/badge/HowToSupport-green)](HowToSupport.md)
+:eu:
+:greece:
 
-[![ko-fi](https://www.ko-fi.com/img/githubbutton_sm.svg)](https://ko-fi.com/andyprogram)
+A program to download photo and video from [any site](#supported-sites) (e.g. YouTube, YouTube Music, OnlyFans, Reddit, Twitter, Mastodon, Instagram, Threads, Facebook, TikTok, RedGifs, JustForFans, PornHub, XHamster, XVIDEOS, ThisVid, LPSG, Pinterest).
+
+**If you like SCrawler, please like the program on [this site](https://alternativeto.net/software/scrawler/about/) and/or [this](https://www.softpedia.com/get/Internet/Download-Managers/Social-networks-crawler.shtml)**
+
+**Join our Discord server**: https://discord.gg/uFNUXvFFmg
+<br/>*If you have problems using the program, you can get help faster on our Discord server!*
+
+<!---Do you like this program? Consider adding to my coffee fund by making a donation to show your support. :blush:
+[![ko-fi](https://www.ko-fi.com/img/githubbutton_sm.svg)](https://ko-fi.com/andyprogram)--->
+**Bitcoin**: BC1Q0NH839FT5TA44DD7L7RLR97XDQAG9V8D6N7XET
+
+[![](https://www.softpedia.com/_img/softpedia_100_free.png?2023_1)](https://www.softpedia.com/get/Internet/Download-Managers/Social-networks-crawler.shtml#status)
+
+![Main window](ProgramScreenshots/MainWindow.png)
+![Channels window](ProgramScreenshots/Channels.png)
+
+[**YouTube standalone application:**](https://github.com/AAndyProgram/SCrawler/wiki/YouTube-downloader)
+
+![YouTube application](ProgramScreenshots/AppYouTube.png)
 
 # What can program do:
-- Download pictures and videos from users' profiles and subreddits:
-  - Reddit images;
-  - Reddit galleries of images;
-  - Redgifs hosted videos (https://www.redgifs.com/);
-  - Reddit hosted videos (downloading Reddit hosted video is going through ffmpeg (**ffmpeg only works with the x64 program**));
-  - Twitter images and videos;
-  - Instagram images and videos.
-- Parse [channel and view data](https://github.com/AAndyProgram/SCrawler/wiki/Channels).
-- Download [saved Reddit and Instagram posts](https://github.com/AAndyProgram/SCrawler/wiki/Home#saved-posts).
-- Add users from parsed channel.
-- Labeling users.
-- Filter exists users by label or group.
+- Download pictures and videos from user profiles:
+  - YouTube videos, shorts, community feeds, users, artists, playlists, music, tracks;
+  - Reddit images, galleries of images, videos, text, saved posts;
+  - Redgifs images and videos (https://www.redgifs.com/);
+  - Twitter images and videos, text, saved (bookmarked) posts, likes, communities;
+  - Bluesky images and videos, text;
+  - OnlyFans images and videos, text, saved (bookmarked) posts, stories;
+  - JustForFans images and videos, saved (bookmarked) posts;
+  - Mastodon images and videos, saved (bookmarked) posts;
+  - Instagram images and videos, text, tagged posts, stories, saved posts;
+  - Threads images and videos, text, saved posts;
+  - Facebook images and videos, stories, saved posts;
+  - TikTok images and videos;
+  - Pinterest boards, users, saved posts;
+  - Imgur images, galleries and videos;
+  - Gfycat videos;
+  - PornHub images, videos, save (liked) posts, search queries, search categories;
+  - XHamster images, videos, saved posts, search queries, search categories, search tags;
+  - XVIDEOS videos, saved posts, search queries, search categories;
+  - ThisVid images, videos, saved posts, search queries, search categories, search tags;
+  - [Other](#supported-sites) supported sites
+- Parse [Reddit channel and view data](https://github.com/AAndyProgram/SCrawler/wiki/Channels)
+- Download [saved posts](https://github.com/AAndyProgram/SCrawler/wiki/Home#saved-posts)
+- Add users from parsed channel
+- **Advanced user management**
+- **Automation** ([downloading data automatically](https://github.com/AAndyProgram/SCrawler/wiki/Settings#automation) every `X` minutes)
+- **Feed** ([feed](https://github.com/AAndyProgram/SCrawler/wiki#feed) of downloaded media files and subscriptions posts)
+- Multiple accounts support
+- Labeling users
+- Create [download groups](https://github.com/AAndyProgram/SCrawler/wiki/Settings#download-groups)
+- Adding users to favorites and temporary
+- Adding users and search queries in the **Subscription** mode (download post preview, but do not download the media file)
+- [Filter exists users](https://github.com/AAndyProgram/SCrawler/wiki#view) by label or group
 - Selection of media types you want to download (images only, videos only, both)
+- [Download a special video](https://github.com/AAndyProgram/SCrawler/wiki#download-separate-video), image or gallery
+- Making [collections](https://github.com/AAndyProgram/SCrawler/wiki#collections) (grouping users into collections)
+- Specifying a user folder (for downloading data to another location)
+- Changing user icons
+- Changing [view modes](https://github.com/AAndyProgram/SCrawler/wiki#view)
+- ...and many others...
 
 # Supported sites
-
-- Reddit
-- Twitter
-- Instagram
+- **YouTube**
+- **YouTube Music**
+- **Reddit**
+- **Twitter**
+- **Bluesky**
+- **OnlyFans** *(partial support)*[^1]
+- **Instagram**
+- **Threads**
+- **Facebook**
+- JustForFans *(partial support) ([video issue](https://discord.com/channels/1124032649682493462/1205547615199039551/1231349555132366870))*[^1]
+- Mastodon *(out of support)*
+- TikTok
 - RedGifs
+- Pinterest
+- Imgur *(out of support)*
+- Gfycat
+- LPSG
+- **PornHub**
+- **XHamster**
+- **XVIDEOS**
+- **ThisVid**
+- [Other sites](Plugins.md)
 
-# How does it works:
+**[SITES REQUIREMENTS](https://github.com/AAndyProgram/SCrawler/wiki/Settings#sites-requirements)**
 
-## Reddit
+# How it works
 
-The program parses all user posts, obtain MD5  images hash and compares them with existing ones to remove duplicates. Then the media will be downloaded.
-
-## Other sites
-
-The program parses all user posts and compares file names with existing ones to remove duplicates. Then the media will be downloaded.
-
-You can read about Instagram restrictions [here](https://github.com/AAndyProgram/SCrawler/wiki/Settings#instagram-limits)
+First, the program downloads the full profile. After the program downloads only new posts. The program remembers downloaded posts.
 
 ## How to request a new site
 
-Read [here](https://github.com/AAndyProgram/SCrawler/blob/main/CONTRIBUTING.md#how-to-request-a-new-site) about
+<!---Read [here](CONTRIBUTING.md#how-to-request-a-new-site) about--->
+**I'm currently not accepting requests to develop new sites.**
 
-# Requirements:
+# Requirements
 
-- Windows 7, 8, 9, 10, 11 with NET Framework 4.6.1 or higher
-- Authorization cookies and tokens for Twitter (if you want to download data from Twitter)
-- Authorization cookies Instagram (if you want to download data from Instagram)
-- ffmpeg library for downloading videos hosted on Reddit (you can download it from the [official repo](https://github.com/GyanD/codexffmpeg/releases/tag/2021-01-12-git-ca21cb1e36) or [from my first release](https://github.com/AAndyProgram/SCrawler/releases/download/1.0.0.0/ffmpeg.zip)). **ffmpeg only works with the x64 version of the program.**
-- **Don't put program in the ```Program Files``` system folder (this is portable program and program settings are stored in the program folder)**
-- **Just unzip the program archive to any folder, copy the file ```ffmpeg.exe``` into it and enjoy. :)**
+- **Windows 10, 11** with NET Framework 4.6.1 or higher (v4.6.1 must be installed). You can check version compatibility with this [tool](Tools/NET.FrameworkVersion.ps1).
+- **[SITES REQUIREMENTS](https://github.com/AAndyProgram/SCrawler/wiki/Settings#sites-requirements)**
 
-# How to build from source
+# Guide
 
-1. Delete the "PersonalUtilities" project from the solution.
-2. Add the latest version of the "PersonalUtilities.dll" library (from the [latest release](https://github.com/AAndyProgram/SCrawler/releases/latest)).
-3. Import PersonalUtilities.Functions for the whole project.
+- [Main window](https://github.com/AAndyProgram/SCrawler/wiki)
+  - [Users](https://github.com/AAndyProgram/SCrawler/wiki/Users)
+    - [Add/Edit/Delete users](https://github.com/AAndyProgram/SCrawler/wiki/Users)
+    - [Collections](https://github.com/AAndyProgram/SCrawler/wiki#collections)
+    - [User operations](https://github.com/AAndyProgram/SCrawler/wiki#context-menu)
+    - [User labels](https://github.com/AAndyProgram/SCrawler/wiki/Users#labels)
+  - **[DOWNLOAD](https://github.com/AAndyProgram/SCrawler/wiki#download)**
+    - [Automation](https://github.com/AAndyProgram/SCrawler/wiki/Settings#automation)
+    - [Download groups](https://github.com/AAndyProgram/SCrawler/wiki/Settings#download-groups)
+  - [Downloading information](https://github.com/AAndyProgram/SCrawler/wiki#info)
+  - [Reddit channels](https://github.com/AAndyProgram/SCrawler/wiki/Channels)
+  - [Saved posts](https://github.com/AAndyProgram/SCrawler/wiki#saved-posts)
+  - [View modes, filters](https://github.com/AAndyProgram/SCrawler/wiki#view)
+- **[SETTINGS](https://github.com/AAndyProgram/SCrawler/wiki/Settings)**
+  - **[SITES REQUIREMENTS](https://github.com/AAndyProgram/SCrawler/wiki/Settings#sites-requirements)**
+  - [Reddit](https://github.com/AAndyProgram/SCrawler/wiki/Settings#reddit)
+  - [Twitter](https://github.com/AAndyProgram/SCrawler/wiki/Settings#twitter)
+  - [Bluesky](https://github.com/AAndyProgram/SCrawler/wiki/Settings#bluesky)
+  - [OnlyFans](https://github.com/AAndyProgram/SCrawler/wiki/Settings#onlyfans)
+  - [Mastodon](https://github.com/AAndyProgram/SCrawler/wiki/Settings#mastodon)
+  - [Instagram](https://github.com/AAndyProgram/SCrawler/wiki/Settings#instagram)
+  - [Threads](https://github.com/AAndyProgram/SCrawler/wiki/Settings#threads)
+  - [Facebook](https://github.com/AAndyProgram/SCrawler/wiki/Settings#facebook)
+  - [JustForFans](https://github.com/AAndyProgram/SCrawler/wiki/Settings#justforfans)
+  - [TikTok](https://github.com/AAndyProgram/SCrawler/wiki/Settings#tiktok)
+  - [RedGifs](https://github.com/AAndyProgram/SCrawler/wiki/Settings#redgifs)
+  - [YouTube](https://github.com/AAndyProgram/SCrawler/wiki/Settings#youtube)
+  - [Pinterest](https://github.com/AAndyProgram/SCrawler/wiki/Settings#Pinterest)
+  - [PornHub](https://github.com/AAndyProgram/SCrawler/wiki/Settings#pornhub)
+  - [XHamster](https://github.com/AAndyProgram/SCrawler/wiki/Settings#xhamster)
+  - [XVIDEOS](https://github.com/AAndyProgram/SCrawler/wiki/Settings#xvideos)
+  - [ThisVid](https://github.com/AAndyProgram/SCrawler/wiki/Settings#thisvid)
+  - [LPSG](https://github.com/AAndyProgram/SCrawler/wiki/Settings#lpsg)
+
+**Full guide you can find [here](https://github.com/AAndyProgram/SCrawler/wiki)**
+
+**Video on how to configure**
+
+[![How to configure](https://img.youtube.com/vi/XDn7zG4I700/0.jpg)](https://www.youtube.com/watch?v=XDn7zG4I700)
+
+# Installation
+
+**Just download the [latest release](https://github.com/AAndyProgram/SCrawler/releases/latest), unzip the program archive to any folder and enjoy.** :blush:
+
+**Don't put program in the `Program Files` system folder (this is portable program and program settings are stored in the program folder)**
+
+**I highly doubt you can run SCrawler on Linux or Mac. SCrawler is a program that is heavily dependent on Windows.**
 
 # Updating
 
-Just download [latest](https://github.com/AAndyProgram/SCrawler/releases/latest) version and unpack it into the program folder. Before starting a new version, I recommend making a backup copy of the program settings folder.
+Just download [latest](https://github.com/AAndyProgram/SCrawler/releases/latest) version and unpack it into the program folder. **Before launching a new version, I recommend making a backup copy of the program settings folder and user settings/data files.**
 
+**You can also use the updater included in the release package.**
 
+# [How to report a problem](CONTRIBUTING.md#how-to-report-a-problem)
+
+# [How to build from source](CONTRIBUTING.md#how-to-build-from-source)
+
+# [How to make a plugin](https://github.com/AAndyProgram/SCrawler/wiki/Plugins)
+
+# [How to support](HowToSupport.md)
 
 # Settings and usage
 
 The program has an intuitive interface.
 
-Just add a user profile and click the ```Start downloading``` button.
+**[SITES REQUIREMENTS](https://github.com/AAndyProgram/SCrawler/wiki/Settings#sites-requirements)**
 
-You can add users by patterns:
-- https://www.instagram.com/SomeUserName
-- https://twitter.com/SomeUserName
-- https://reddit.com/user/SomeUserName
-- https://reddit.com/r/SomeSubredditName
-- https://www.redgifs.com/users/SomeUserName
-- u/SomeUserName
-- r/SomeSubredditName
-- SomeUserName (in this case, you need to select the user's site)
-- SomeSubredditName
+[![How to configure](https://img.youtube.com/vi/XDn7zG4I700/0.jpg)](https://www.youtube.com/watch?v=XDn7zG4I700)
 
-Read more about adding users and subreddits [here](https://github.com/AAndyProgram/SCrawler/wiki/Users)
+Just add a user profile and **click the `Download` button**.
 
-**Full guide you can find [here](https://github.com/AAndyProgram/SCrawler/wiki)**
+```mermaid
+stateDiagram
+Start: Add site credentials
+What: What would I like to do
+DownUser: Download user
+DownVideo: Download video
+AUser: Add user (1)
+OVIF: Open standalone downloader (2)
+AVideo: Add video url
+F5: Press 'F5' or click the download button
+[*]-->Start
+Start-->What
+What-->DownUser
+What-->DownVideo
+DownUser-->AUser
+DownVideo-->OVIF
+OVIF-->AVideo
+AVideo-->F5
+AUser-->F5
+F5-->[*]
+```
+1. Press `Insert` or click the `Download` button ([read more here](https://github.com/AAndyProgram/SCrawler/wiki#users-list), [hot keys](https://github.com/AAndyProgram/SCrawler/wiki#hot-keys))
+2. Click the `Download` button, then `Standalone downloader` ([read more here](https://github.com/AAndyProgram/SCrawler/wiki#download-separate-video))
 
-## Using program as just video downloader
-
-Create a shortcut for the program. Open shortcut properties. In the ```Shortcut``` tab, in the ```Target``` field, just add the letter ```v``` at the end across the space.
-
-Example: ```D:\Programs\SCrawler\SCrawler.exe v```
+![Add user](ProgramScreenshots/CreateUserClear.png)
 
 # Contact me
 
-[Element messenger](https://element.io/): @andyprogram:matrix.org
-https://matrix.to/#/@andyprogram:matrix.org
+Discord server: https://discord.gg/uFNUXvFFmg
+
+[^1]: Partial support means that I don't have personal accounts on paid porn sites because I don't pay for porn. If this site has stopped downloading and you want me to fix it, please be ready to give me access to an account with at least one active subscription. Otherwise, the download from this site will not be fixed.

SCrawler.Plugin.LPSG/.editorconfig

@@ -0,0 +1,3 @@
+[*.vb]
+# Modifier preferences
+file_header_template = Copyright (C) 2022  Andy\nThis program is free software: you can redistribute it and/or modify\nit under the terms of the GNU General Public License as published by\nthe Free Software Foundation, either version 3 of the License, or\n(at your option) any later version.\n\nThis program is distributed in the hope that it will be useful,\nbut WITHOUT ANY WARRANTY; without even the implied warranty of\nMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the\nGNU General Public License for more details.\n\nYou should have received a copy of the GNU General Public License\nalong with this program.  If not, see <https://www.gnu.org/licenses/>

SCrawler.Plugin.LPSG/Declarations.vb

@@ -0,0 +1,34 @@
+' Copyright (C) 2022  Andy
+' This program is free software: you can redistribute it and/or modify
+' it under the terms of the GNU General Public License as published by
+' the Free Software Foundation, either version 3 of the License, or
+' (at your option) any later version.
+'
+' This program is distributed in the hope that it will be useful,
+' but WITHOUT ANY WARRANTY
+Imports PersonalUtilities.Functions.RegularExpressions
+Friend Module Declarations
+    Friend ReadOnly Property PhotoRegEx As RParams = RParams.DM("(https://www.lpsg.com/attachments)(.+?)(?="")", 0, RegexReturn.List)
+    Friend ReadOnly Property PhotoRegExExt As New RParams("img.data.src=""(/proxy[^""]+?)""", Nothing, 1, RegexReturn.List) With {
+        .Converter = Function(Input) $"https://www.lpsg.com/{SymbolsConverter.HTML.Decode(Input)}"}
+    Friend ReadOnly Property NextPageRegex As RParams = RParams.DMS("<link rel=""next"" href=""(.+?/page-(\d+))""", 2)
+    Private Const FileUrlRegexDefault As String = "([^/]+?)(jpg|jpeg|gif|png|webm)"
+    Private ReadOnly InputFReplacer As New ErrorsDescriber(EDP.ReturnValue)
+    Private ReadOnly InputForbidRemover As Func(Of String, String) = Function(Input) If(Input.IsEmptyString, Input, Input.StringRemoveWinForbiddenSymbols(, InputFReplacer))
+    Friend ReadOnly Property FileRegEx As New RParams(FileUrlRegexDefault, Nothing, 0) With {
+        .Converter = Function(ByVal Input As String) As String
+                         Input = InputForbidRemover.Invoke(Input)
+                         If Not Input.IsEmptyString Then
+                             Dim lv$ = Input.Split("-").LastOrDefault
+                             If Not lv.IsEmptyString Then
+                                 Input = Input.Replace($"-{lv}", String.Empty)
+                                 Input &= $".{lv}"
+                             End If
+                         End If
+                         Return Input
+                     End Function}
+    Friend ReadOnly Property FileRegExExt As New RParams(FileUrlRegexDefault, 0, Nothing, InputForbidRemover)
+    Friend ReadOnly Property FileRegExExt2 As New RParams("([^/]+?)(?=(\Z|&))", 0, Nothing, InputForbidRemover)
+    Friend ReadOnly Property FileExistsRegEx As RParams = RParams.DMS(FileUrlRegexDefault, 2)
+    Friend ReadOnly TempListAddParams As New ListAddParams(LAP.NotContainsOnly) With {.Comparer = New FComparer(Of PluginUserMedia)(Function(x, y) x.URL = y.URL)}
+End Module

SCrawler.Plugin.LPSG/My Project/Application.Designer.vb

@@ -0,0 +1,13 @@
+'------------------------------------------------------------------------------
+' <auto-generated>
+'     This code was generated by a tool.
+'     Runtime Version:4.0.30319.42000
+'
+'     Changes to this file may cause incorrect behavior and will be lost if
+'     the code is regenerated.
+' </auto-generated>
+'------------------------------------------------------------------------------
+
+Option Strict On
+Option Explicit On
+

SCrawler.Plugin.LPSG/My Project/Application.myapp

@@ -0,0 +1,10 @@
+<?xml version="1.0" encoding="utf-8"?>
+<MyApplicationData xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
+  <MySubMain>false</MySubMain>
+  <SingleInstance>false</SingleInstance>
+  <ShutdownMode>0</ShutdownMode>
+  <EnableVisualStyles>true</EnableVisualStyles>
+  <AuthenticationMode>0</AuthenticationMode>
+  <ApplicationType>1</ApplicationType>
+  <SaveMySettingsOnExit>true</SaveMySettingsOnExit>
+</MyApplicationData>

SCrawler.Plugin.LPSG/My Project/AssemblyInfo.vb

@@ -0,0 +1,37 @@
+Imports System.Resources
+Imports System
+Imports System.Reflection
+Imports System.Runtime.InteropServices
+
+' General Information about an assembly is controlled through the following
+' set of attributes. Change these attribute values to modify the information
+' associated with an assembly.
+
+' Review the values of the assembly attributes
+
+<Assembly: AssemblyTitle("LPSG")>
+<Assembly: AssemblyDescription("LPSG plugin for SCrawler")>
+<Assembly: AssemblyCompany("AndyProgram")>
+<Assembly: AssemblyProduct("LPSG")>
+<Assembly: AssemblyCopyright("Copyright © 2022")>
+<Assembly: AssemblyTrademark("AndyProgram")>
+
+<Assembly: ComVisible(False)>
+
+'The following GUID is for the ID of the typelib if this project is exposed to COM
+<Assembly: Guid("4beaf92d-0566-4331-ad29-034d09a7326f")>
+
+' Version information for an assembly consists of the following four values:
+'
+'      Major Version
+'      Minor Version
+'      Build Number
+'      Revision
+'
+' You can specify all the values or you can default the Build and Revision Numbers
+' by using the '*' as shown below:
+' <Assembly: AssemblyVersion("1.0.*")>
+
+<Assembly: AssemblyVersion("2022.10.12.0")>
+<Assembly: AssemblyFileVersion("2022.10.12.0")>
+<Assembly: NeutralResourcesLanguage("en")>

SCrawler.Plugin.LPSG/My Project/Resources.Designer.vb

@@ -0,0 +1,83 @@
+'------------------------------------------------------------------------------
+' <auto-generated>
+'     This code was generated by a tool.
+'     Runtime Version:4.0.30319.42000
+'
+'     Changes to this file may cause incorrect behavior and will be lost if
+'     the code is regenerated.
+' </auto-generated>
+'------------------------------------------------------------------------------
+
+Option Strict On
+Option Explicit On
+
+Imports System
+
+Namespace My.Resources
+    
+    'This class was auto-generated by the StronglyTypedResourceBuilder
+    'class via a tool like ResGen or Visual Studio.
+    'To add or remove a member, edit your .ResX file then rerun ResGen
+    'with the /str option, or rebuild your VS project.
+    '''<summary>
+    '''  A strongly-typed resource class, for looking up localized strings, etc.
+    '''</summary>
+    <Global.System.CodeDom.Compiler.GeneratedCodeAttribute("System.Resources.Tools.StronglyTypedResourceBuilder", "16.0.0.0"),  _
+     Global.System.Diagnostics.DebuggerNonUserCodeAttribute(),  _
+     Global.System.Runtime.CompilerServices.CompilerGeneratedAttribute(),  _
+     Global.Microsoft.VisualBasic.HideModuleNameAttribute()>  _
+    Friend Module Resources
+        
+        Private resourceMan As Global.System.Resources.ResourceManager
+        
+        Private resourceCulture As Global.System.Globalization.CultureInfo
+        
+        '''<summary>
+        '''  Returns the cached ResourceManager instance used by this class.
+        '''</summary>
+        <Global.System.ComponentModel.EditorBrowsableAttribute(Global.System.ComponentModel.EditorBrowsableState.Advanced)>  _
+        Friend ReadOnly Property ResourceManager() As Global.System.Resources.ResourceManager
+            Get
+                If Object.ReferenceEquals(resourceMan, Nothing) Then
+                    Dim temp As Global.System.Resources.ResourceManager = New Global.System.Resources.ResourceManager("LPSG.Resources", GetType(Resources).Assembly)
+                    resourceMan = temp
+                End If
+                Return resourceMan
+            End Get
+        End Property
+        
+        '''<summary>
+        '''  Overrides the current thread's CurrentUICulture property for all
+        '''  resource lookups using this strongly typed resource class.
+        '''</summary>
+        <Global.System.ComponentModel.EditorBrowsableAttribute(Global.System.ComponentModel.EditorBrowsableState.Advanced)>  _
+        Friend Property Culture() As Global.System.Globalization.CultureInfo
+            Get
+                Return resourceCulture
+            End Get
+            Set
+                resourceCulture = value
+            End Set
+        End Property
+        
+        '''<summary>
+        '''  Looks up a localized resource of type System.Drawing.Icon similar to (Icon).
+        '''</summary>
+        Friend ReadOnly Property Icon32() As System.Drawing.Icon
+            Get
+                Dim obj As Object = ResourceManager.GetObject("Icon32", resourceCulture)
+                Return CType(obj,System.Drawing.Icon)
+            End Get
+        End Property
+        
+        '''<summary>
+        '''  Looks up a localized resource of type System.Drawing.Bitmap.
+        '''</summary>
+        Friend ReadOnly Property IconPic_32() As System.Drawing.Bitmap
+            Get
+                Dim obj As Object = ResourceManager.GetObject("IconPic_32", resourceCulture)
+                Return CType(obj,System.Drawing.Bitmap)
+            End Get
+        End Property
+    End Module
+End Namespace

SCrawler.Plugin.LPSG/My Project/Resources.resx

@@ -0,0 +1,127 @@
+<?xml version="1.0" encoding="utf-8"?>
+<root>
+  <!-- 
+    Microsoft ResX Schema 
+    
+    Version 2.0
+    
+    The primary goals of this format is to allow a simple XML format 
+    that is mostly human readable. The generation and parsing of the 
+    various data types are done through the TypeConverter classes 
+    associated with the data types.
+    
+    Example:
+    
+    ... ado.net/XML headers & schema ...
+    <resheader name="resmimetype">text/microsoft-resx</resheader>
+    <resheader name="version">2.0</resheader>
+    <resheader name="reader">System.Resources.ResXResourceReader, System.Windows.Forms, ...</resheader>
+    <resheader name="writer">System.Resources.ResXResourceWriter, System.Windows.Forms, ...</resheader>
+    <data name="Name1"><value>this is my long string</value><comment>this is a comment</comment></data>
+    <data name="Color1" type="System.Drawing.Color, System.Drawing">Blue</data>
+    <data name="Bitmap1" mimetype="application/x-microsoft.net.object.binary.base64">
+        <value>[base64 mime encoded serialized .NET Framework object]</value>
+    </data>
+    <data name="Icon1" type="System.Drawing.Icon, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+        <value>[base64 mime encoded string representing a byte array form of the .NET Framework object]</value>
+        <comment>This is a comment</comment>
+    </data>
+                
+    There are any number of "resheader" rows that contain simple 
+    name/value pairs.
+    
+    Each data row contains a name, and value. The row also contains a 
+    type or mimetype. Type corresponds to a .NET class that support 
+    text/value conversion through the TypeConverter architecture. 
+    Classes that don't support this are serialized and stored with the 
+    mimetype set.
+    
+    The mimetype is used for serialized objects, and tells the 
+    ResXResourceReader how to depersist the object. This is currently not 
+    extensible. For a given mimetype the value must be set accordingly:
+    
+    Note - application/x-microsoft.net.object.binary.base64 is the format 
+    that the ResXResourceWriter will generate, however the reader can 
+    read any of the formats listed below.
+    
+    mimetype: application/x-microsoft.net.object.binary.base64
+    value   : The object must be serialized with 
+            : System.Runtime.Serialization.Formatters.Binary.BinaryFormatter
+            : and then encoded with base64 encoding.
+    
+    mimetype: application/x-microsoft.net.object.soap.base64
+    value   : The object must be serialized with 
+            : System.Runtime.Serialization.Formatters.Soap.SoapFormatter
+            : and then encoded with base64 encoding.
+
+    mimetype: application/x-microsoft.net.object.bytearray.base64
+    value   : The object must be serialized into a byte array 
+            : using a System.ComponentModel.TypeConverter
+            : and then encoded with base64 encoding.
+    -->
+  <xsd:schema id="root" xmlns="" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:msdata="urn:schemas-microsoft-com:xml-msdata">
+    <xsd:import namespace="http://www.w3.org/XML/1998/namespace" />
+    <xsd:element name="root" msdata:IsDataSet="true">
+      <xsd:complexType>
+        <xsd:choice maxOccurs="unbounded">
+          <xsd:element name="metadata">
+            <xsd:complexType>
+              <xsd:sequence>
+                <xsd:element name="value" type="xsd:string" minOccurs="0" />
+              </xsd:sequence>
+              <xsd:attribute name="name" use="required" type="xsd:string" />
+              <xsd:attribute name="type" type="xsd:string" />
+              <xsd:attribute name="mimetype" type="xsd:string" />
+              <xsd:attribute ref="xml:space" />
+            </xsd:complexType>
+          </xsd:element>
+          <xsd:element name="assembly">
+            <xsd:complexType>
+              <xsd:attribute name="alias" type="xsd:string" />
+              <xsd:attribute name="name" type="xsd:string" />
+            </xsd:complexType>
+          </xsd:element>
+          <xsd:element name="data">
+            <xsd:complexType>
+              <xsd:sequence>
+                <xsd:element name="value" type="xsd:string" minOccurs="0" msdata:Ordinal="1" />
+                <xsd:element name="comment" type="xsd:string" minOccurs="0" msdata:Ordinal="2" />
+              </xsd:sequence>
+              <xsd:attribute name="name" type="xsd:string" use="required" msdata:Ordinal="1" />
+              <xsd:attribute name="type" type="xsd:string" msdata:Ordinal="3" />
+              <xsd:attribute name="mimetype" type="xsd:string" msdata:Ordinal="4" />
+              <xsd:attribute ref="xml:space" />
+            </xsd:complexType>
+          </xsd:element>
+          <xsd:element name="resheader">
+            <xsd:complexType>
+              <xsd:sequence>
+                <xsd:element name="value" type="xsd:string" minOccurs="0" msdata:Ordinal="1" />
+              </xsd:sequence>
+              <xsd:attribute name="name" type="xsd:string" use="required" />
+            </xsd:complexType>
+          </xsd:element>
+        </xsd:choice>
+      </xsd:complexType>
+    </xsd:element>
+  </xsd:schema>
+  <resheader name="resmimetype">
+    <value>text/microsoft-resx</value>
+  </resheader>
+  <resheader name="version">
+    <value>2.0</value>
+  </resheader>
+  <resheader name="reader">
+    <value>System.Resources.ResXResourceReader, System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
+  </resheader>
+  <resheader name="writer">
+    <value>System.Resources.ResXResourceWriter, System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
+  </resheader>
+  <assembly alias="System.Windows.Forms" name="System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" />
+  <data name="Icon32" type="System.Resources.ResXFileRef, System.Windows.Forms">
+    <value>..\Content\Icons\Icon32.ico;System.Drawing.Icon, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a</value>
+  </data>
+  <data name="IconPic_32" type="System.Resources.ResXFileRef, System.Windows.Forms">
+    <value>..\Content\Pictures\IconPic_32.png;System.Drawing.Bitmap, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a</value>
+  </data>
+</root>

SCrawler.Plugin.LPSG/My Project/Settings.Designer.vb

@@ -0,0 +1,73 @@
+'------------------------------------------------------------------------------
+' <auto-generated>
+'     This code was generated by a tool.
+'     Runtime Version:4.0.30319.42000
+'
+'     Changes to this file may cause incorrect behavior and will be lost if
+'     the code is regenerated.
+' </auto-generated>
+'------------------------------------------------------------------------------
+
+Option Strict On
+Option Explicit On
+
+
+Namespace My
+    
+    <Global.System.Runtime.CompilerServices.CompilerGeneratedAttribute(),  _
+     Global.System.CodeDom.Compiler.GeneratedCodeAttribute("Microsoft.VisualStudio.Editors.SettingsDesigner.SettingsSingleFileGenerator", "16.10.0.0"),  _
+     Global.System.ComponentModel.EditorBrowsableAttribute(Global.System.ComponentModel.EditorBrowsableState.Advanced)>  _
+    Partial Friend NotInheritable Class MySettings
+        Inherits Global.System.Configuration.ApplicationSettingsBase
+        
+        Private Shared defaultInstance As MySettings = CType(Global.System.Configuration.ApplicationSettingsBase.Synchronized(New MySettings()),MySettings)
+        
+#Region "My.Settings Auto-Save Functionality"
+#If _MyType = "WindowsForms" Then
+    Private Shared addedHandler As Boolean
+
+    Private Shared addedHandlerLockObject As New Object
+
+    <Global.System.Diagnostics.DebuggerNonUserCodeAttribute(), Global.System.ComponentModel.EditorBrowsableAttribute(Global.System.ComponentModel.EditorBrowsableState.Advanced)> _
+    Private Shared Sub AutoSaveSettings(sender As Global.System.Object, e As Global.System.EventArgs)
+        If My.Application.SaveMySettingsOnExit Then
+            My.Settings.Save()
+        End If
+    End Sub
+#End If
+#End Region
+        
+        Public Shared ReadOnly Property [Default]() As MySettings
+            Get
+                
+#If _MyType = "WindowsForms" Then
+               If Not addedHandler Then
+                    SyncLock addedHandlerLockObject
+                        If Not addedHandler Then
+                            AddHandler My.Application.Shutdown, AddressOf AutoSaveSettings
+                            addedHandler = True
+                        End If
+                    End SyncLock
+                End If
+#End If
+                Return defaultInstance
+            End Get
+        End Property
+    End Class
+End Namespace
+
+Namespace My
+    
+    <Global.Microsoft.VisualBasic.HideModuleNameAttribute(),  _
+     Global.System.Diagnostics.DebuggerNonUserCodeAttribute(),  _
+     Global.System.Runtime.CompilerServices.CompilerGeneratedAttribute()>  _
+    Friend Module MySettingsProperty
+        
+        <Global.System.ComponentModel.Design.HelpKeywordAttribute("My.Settings")>  _
+        Friend ReadOnly Property Settings() As Global.LPSG.My.MySettings
+            Get
+                Return Global.LPSG.My.MySettings.Default
+            End Get
+        End Property
+    End Module
+End Namespace

SCrawler.Plugin.LPSG/My Project/Settings.settings

@@ -0,0 +1,7 @@
+<?xml version='1.0' encoding='utf-8'?>
+<SettingsFile xmlns="http://schemas.microsoft.com/VisualStudio/2004/01/settings" CurrentProfile="(Default)" UseMySettingsClassName="true">
+  <Profiles>
+    <Profile Name="(Default)" />
+  </Profiles>
+  <Settings />
+</SettingsFile>

SCrawler.Plugin.LPSG/SCrawler.Plugin.LPSG.vbproj

@@ -0,0 +1,166 @@
+<?xml version="1.0" encoding="utf-8"?>
+<Project ToolsVersion="15.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
+  <Import Project="$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props" Condition="Exists('$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props')" />
+  <PropertyGroup>
+    <Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
+    <Platform Condition=" '$(Platform)' == '' ">AnyCPU</Platform>
+    <ProjectGuid>{22A130B2-DDF4-4FB5-BA38-E5DB4CF1B8A2}</ProjectGuid>
+    <OutputType>Library</OutputType>
+    <RootNamespace>LPSG</RootNamespace>
+    <AssemblyName>SCrawler.Plugin.LPSG</AssemblyName>
+    <FileAlignment>512</FileAlignment>
+    <MyType>Windows</MyType>
+    <TargetFrameworkVersion>v4.6.1</TargetFrameworkVersion>
+    <Deterministic>true</Deterministic>
+  </PropertyGroup>
+  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
+    <DebugSymbols>true</DebugSymbols>
+    <DebugType>full</DebugType>
+    <DefineDebug>true</DefineDebug>
+    <DefineTrace>true</DefineTrace>
+    <OutputPath>bin\Debug\</OutputPath>
+    <DocumentationFile>
+    </DocumentationFile>
+    <NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
+  </PropertyGroup>
+  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
+    <DebugType>pdbonly</DebugType>
+    <DefineDebug>false</DefineDebug>
+    <DefineTrace>true</DefineTrace>
+    <Optimize>true</Optimize>
+    <OutputPath>bin\Release\</OutputPath>
+    <DocumentationFile>
+    </DocumentationFile>
+    <NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
+  </PropertyGroup>
+  <PropertyGroup>
+    <OptionExplicit>On</OptionExplicit>
+  </PropertyGroup>
+  <PropertyGroup>
+    <OptionCompare>Binary</OptionCompare>
+  </PropertyGroup>
+  <PropertyGroup>
+    <OptionStrict>Off</OptionStrict>
+  </PropertyGroup>
+  <PropertyGroup>
+    <OptionInfer>On</OptionInfer>
+  </PropertyGroup>
+  <PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Debug|x64'">
+    <DebugSymbols>true</DebugSymbols>
+    <DefineDebug>true</DefineDebug>
+    <DefineTrace>true</DefineTrace>
+    <OutputPath>bin\x64\Debug\</OutputPath>
+    <NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
+    <DebugType>full</DebugType>
+    <PlatformTarget>x64</PlatformTarget>
+  </PropertyGroup>
+  <PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Release|x64'">
+    <DefineTrace>true</DefineTrace>
+    <OutputPath>bin\x64\Release\</OutputPath>
+    <Optimize>true</Optimize>
+    <NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
+    <DebugType>pdbonly</DebugType>
+    <PlatformTarget>x64</PlatformTarget>
+  </PropertyGroup>
+  <PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Debug|x86'">
+    <DebugSymbols>true</DebugSymbols>
+    <DefineDebug>true</DefineDebug>
+    <DefineTrace>true</DefineTrace>
+    <OutputPath>bin\x86\Debug\</OutputPath>
+    <NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
+    <DebugType>full</DebugType>
+    <PlatformTarget>x86</PlatformTarget>
+  </PropertyGroup>
+  <PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Release|x86'">
+    <DefineTrace>true</DefineTrace>
+    <OutputPath>bin\x86\Release\</OutputPath>
+    <Optimize>true</Optimize>
+    <NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
+    <DebugType>pdbonly</DebugType>
+    <PlatformTarget>x86</PlatformTarget>
+  </PropertyGroup>
+  <ItemGroup>
+    <Reference Include="System" />
+    <Reference Include="System.Data" />
+    <Reference Include="System.Drawing" />
+    <Reference Include="System.Windows.Forms" />
+    <Reference Include="System.Xml" />
+    <Reference Include="System.Core" />
+    <Reference Include="System.Xml.Linq" />
+    <Reference Include="System.Data.DataSetExtensions" />
+    <Reference Include="System.Net.Http" />
+  </ItemGroup>
+  <ItemGroup>
+    <Import Include="Microsoft.VisualBasic" />
+    <Import Include="PersonalUtilities.Functions" />
+    <Import Include="PersonalUtilities.Tools.WEB" />
+    <Import Include="SCrawler.Plugin" />
+    <Import Include="SCrawler.Plugin.Attributes" />
+    <Import Include="System" />
+    <Import Include="System.Collections" />
+    <Import Include="System.Collections.Generic" />
+    <Import Include="System.Data" />
+    <Import Include="System.Diagnostics" />
+    <Import Include="System.Linq" />
+    <Import Include="System.Xml.Linq" />
+    <Import Include="System.Threading.Tasks" />
+  </ItemGroup>
+  <ItemGroup>
+    <Compile Include="Declarations.vb" />
+    <Compile Include="My Project\AssemblyInfo.vb" />
+    <Compile Include="My Project\Application.Designer.vb">
+      <AutoGen>True</AutoGen>
+      <DependentUpon>Application.myapp</DependentUpon>
+      <DesignTime>True</DesignTime>
+    </Compile>
+    <Compile Include="My Project\Resources.Designer.vb">
+      <AutoGen>True</AutoGen>
+      <DesignTime>True</DesignTime>
+      <DependentUpon>Resources.resx</DependentUpon>
+    </Compile>
+    <Compile Include="My Project\Settings.Designer.vb">
+      <AutoGen>True</AutoGen>
+      <DependentUpon>Settings.settings</DependentUpon>
+      <DesignTimeSharedInput>True</DesignTimeSharedInput>
+    </Compile>
+    <Compile Include="SiteSettings.vb" />
+    <Compile Include="UserData.vb" />
+  </ItemGroup>
+  <ItemGroup>
+    <EmbeddedResource Include="My Project\Resources.resx">
+      <Generator>VbMyResourcesResXFileCodeGenerator</Generator>
+      <LastGenOutput>Resources.Designer.vb</LastGenOutput>
+      <CustomToolNamespace>My.Resources</CustomToolNamespace>
+      <SubType>Designer</SubType>
+    </EmbeddedResource>
+  </ItemGroup>
+  <ItemGroup>
+    <None Include=".editorconfig" />
+    <None Include="My Project\Application.myapp">
+      <Generator>MyApplicationCodeGenerator</Generator>
+      <LastGenOutput>Application.Designer.vb</LastGenOutput>
+    </None>
+    <None Include="My Project\Settings.settings">
+      <Generator>SettingsSingleFileGenerator</Generator>
+      <CustomToolNamespace>My</CustomToolNamespace>
+      <LastGenOutput>Settings.Designer.vb</LastGenOutput>
+    </None>
+  </ItemGroup>
+  <ItemGroup>
+    <ProjectReference Include="..\..\..\MyUtilities\PersonalUtilities\PersonalUtilities.vbproj">
+      <Project>{8405896b-2685-4916-bc93-1fb514c323a9}</Project>
+      <Name>PersonalUtilities</Name>
+    </ProjectReference>
+    <ProjectReference Include="..\SCrawler.PluginProvider\SCrawler.PluginProvider.vbproj">
+      <Project>{d4650f6b-5a54-44b6-999b-6c675b7116b1}</Project>
+      <Name>SCrawler.PluginProvider</Name>
+    </ProjectReference>
+  </ItemGroup>
+  <ItemGroup>
+    <None Include="Content\Icons\Icon32.ico" />
+  </ItemGroup>
+  <ItemGroup>
+    <None Include="Content\Pictures\IconPic_32.png" />
+  </ItemGroup>
+  <Import Project="$(MSBuildToolsPath)\Microsoft.VisualBasic.targets" />
+</Project>

SCrawler.Plugin.LPSG/SiteSettings.vb

@@ -0,0 +1,107 @@
+' Copyright (C) 2022  Andy
+' This program is free software: you can redistribute it and/or modify
+' it under the terms of the GNU General Public License as published by
+' the Free Software Foundation, either version 3 of the License, or
+' (at your option) any later version.
+'
+' This program is distributed in the hope that it will be useful,
+' but WITHOUT ANY WARRANTY
+Imports System.Drawing
+Imports PersonalUtilities.Functions.RegularExpressions
+<Manifest("AndyProgram_LPSG")>
+Public Class SiteSettings : Implements ISiteSettings
+    Public ReadOnly Property Icon As Icon Implements ISiteSettings.Icon
+        Get
+            Return My.Resources.Icon32
+        End Get
+    End Property
+    Public ReadOnly Property Image As Image Implements ISiteSettings.Image
+        Get
+            Return My.Resources.IconPic_32
+        End Get
+    End Property
+    Public ReadOnly Property Site As String = "LPSG" Implements ISiteSettings.Site
+    Public Property Logger As ILogProvider Implements ISiteSettings.Logger
+    Friend ReadOnly Property Responser As Response
+    Public Sub New()
+        Responser = New Response($"Settings\Responser_LPSG.xml")
+        With Responser
+            If .File.Exists Then
+                .LoadSettings()
+            Else
+                .CookiesDomain = "www.lpsg.com"
+                .Cookies = New CookieKeeper(.CookiesDomain)
+            End If
+        End With
+    End Sub
+    Public Sub BeginInit() Implements ISiteSettings.BeginInit
+    End Sub
+    Public Sub EndInit() Implements ISiteSettings.EndInit
+    End Sub
+    Public Function GetInstance(ByVal What As ISiteSettings.Download) As IPluginContentProvider Implements ISiteSettings.GetInstance
+        Return New UserData
+    End Function
+    Public Sub Load(ByVal XMLValues As IEnumerable(Of KeyValuePair(Of String, String))) Implements ISiteSettings.Load
+    End Sub
+#Region "Download functions"
+    Public Sub DownloadStarted(ByVal What As ISiteSettings.Download) Implements ISiteSettings.DownloadStarted
+    End Sub
+    Public Sub BeforeStartDownload(ByVal User As Object, ByVal What As ISiteSettings.Download) Implements ISiteSettings.BeforeStartDownload
+    End Sub
+    Public Sub AfterDownload(ByVal User As Object, ByVal What As ISiteSettings.Download) Implements ISiteSettings.AfterDownload
+    End Sub
+    Public Sub DownloadDone(ByVal What As ISiteSettings.Download) Implements ISiteSettings.DownloadDone
+    End Sub
+#End Region
+#Region "Update"
+    Public Sub BeginEdit() Implements ISiteSettings.BeginEdit
+    End Sub
+    Public Sub EndEdit() Implements ISiteSettings.EndEdit
+    End Sub
+    Public Sub BeginUpdate() Implements ISiteSettings.BeginUpdate
+    End Sub
+    Public Sub EndUpdate() Implements ISiteSettings.EndUpdate
+    End Sub
+    Public Sub Update() Implements ISiteSettings.Update
+        Responser.SaveSettings()
+    End Sub
+#End Region
+    Public Sub Reset() Implements ISiteSettings.Reset
+    End Sub
+    Public Sub OpenSettingsForm() Implements ISiteSettings.OpenSettingsForm
+    End Sub
+    Public Sub UserOptions(ByRef Options As Object, ByVal OpenForm As Boolean) Implements ISiteSettings.UserOptions
+        Options = Nothing
+    End Sub
+    Public Function GetUserUrl(ByVal UserName As String, ByVal Channel As Boolean) As String Implements ISiteSettings.GetUserUrl
+        Return $"https://www.lpsg.com/threads/{UserName}/"
+    End Function
+    Private ReadOnly UserRegEx As RParams = RParams.DMS(".+?lpsg.com/threads/([^/]+)", 1)
+    Public Function IsMyUser(ByVal UserURL As String) As ExchangeOptions Implements ISiteSettings.IsMyUser
+        Try
+            Dim r$ = RegexReplace(UserURL, UserRegEx)
+            If Not r.IsEmptyString Then
+                Return New ExchangeOptions(Site, r)
+            Else
+                Return Nothing
+            End If
+        Catch
+            Return Nothing
+        End Try
+    End Function
+    Public Function IsMyImageVideo(ByVal URL As String) As ExchangeOptions Implements ISiteSettings.IsMyImageVideo
+        Return Nothing
+    End Function
+    Public Function GetSpecialData(ByVal URL As String, ByVal Path As String, ByVal AskForPath As Boolean) As IEnumerable(Of PluginUserMedia) Implements ISiteSettings.GetSpecialData
+        Return Nothing
+    End Function
+    Public Function Available(ByVal What As ISiteSettings.Download, ByVal Silent As Boolean) As Boolean Implements ISiteSettings.Available
+        Return If(Responser.Cookies?.Count, 0) > 0
+    End Function
+    Public Function ReadyToDownload(ByVal What As ISiteSettings.Download) As Boolean Implements ISiteSettings.ReadyToDownload
+        Return True
+    End Function
+    Public Function GetUserPostUrl(ByVal UserID As String, ByVal PostID As String) As String Implements ISiteSettings.GetUserPostUrl
+        Return String.Empty
+    End Function
+End Class

SCrawler.Plugin.LPSG/UserData.vb

@@ -0,0 +1,192 @@
+' Copyright (C) 2022  Andy
+' This program is free software: you can redistribute it and/or modify
+' it under the terms of the GNU General Public License as published by
+' the Free Software Foundation, either version 3 of the License, or
+' (at your option) any later version.
+'
+' This program is distributed in the hope that it will be useful,
+' but WITHOUT ANY WARRANTY
+Imports PersonalUtilities.Functions.RegularExpressions
+Imports UStates = SCrawler.Plugin.PluginUserMedia.States
+Imports UTypes = SCrawler.Plugin.PluginUserMedia.Types
+Imports Converters = PersonalUtilities.Functions.SymbolsConverter.Converters
+Public Class UserData : Implements IPluginContentProvider
+#Region "XML names"
+    Private Const Name_LatestPage As String = "LatestPage"
+#End Region
+#Region "Interface declarations"
+    Public Event ProgressChanged(ByVal Count As Integer) Implements IPluginContentProvider.ProgressChanged
+    Public Event TotalCountChanged(ByVal Count As Integer) Implements IPluginContentProvider.TotalCountChanged
+    Public Property Thrower As IThrower Implements IPluginContentProvider.Thrower
+    Public Property LogProvider As ILogProvider Implements IPluginContentProvider.LogProvider
+    Public Property ESettings As ISiteSettings Implements IPluginContentProvider.Settings
+    Private ReadOnly Property Settings As SiteSettings
+        Get
+            Return DirectCast(ESettings, SiteSettings)
+        End Get
+    End Property
+    Public Property Name As String Implements IPluginContentProvider.Name
+    Public Property ID As String Implements IPluginContentProvider.ID
+    Public Property ParseUserMediaOnly As Boolean Implements IPluginContentProvider.ParseUserMediaOnly
+    Public Property UserDescription As String Implements IPluginContentProvider.UserDescription
+    Public Property ExistingContentList As List(Of PluginUserMedia) Implements IPluginContentProvider.ExistingContentList
+    Public Property TempPostsList As List(Of String) Implements IPluginContentProvider.TempPostsList
+    Public Property TempMediaList As List(Of PluginUserMedia) Implements IPluginContentProvider.TempMediaList
+    Public Property UserExists As Boolean Implements IPluginContentProvider.UserExists
+    Public Property UserSuspended As Boolean Implements IPluginContentProvider.UserSuspended
+    Public Property IsSavedPosts As Boolean Implements IPluginContentProvider.IsSavedPosts
+    Public Property SeparateVideoFolder As Boolean Implements IPluginContentProvider.SeparateVideoFolder
+    Public Property DataPath As String Implements IPluginContentProvider.DataPath
+    Public Property PostsNumberLimit As Integer? Implements IPluginContentProvider.PostsNumberLimit
+    Public Property DownloadDateFrom As Date? Implements IPluginContentProvider.DownloadDateFrom
+    Public Property DownloadDateTo As Date? Implements IPluginContentProvider.DownloadDateTo
+#End Region
+#Region "Interface exchange options"
+    Public Sub ExchangeOptionsSet(ByVal Obj As Object) Implements IPluginContentProvider.ExchangeOptionsSet
+    End Sub
+    Public Function ExchangeOptionsGet() As Object Implements IPluginContentProvider.ExchangeOptionsGet
+        Return Nothing
+    End Function
+#End Region
+#Region "Interface XML"
+    Public Sub XmlFieldsSet(ByVal Fields As List(Of KeyValuePair(Of String, String))) Implements IPluginContentProvider.XmlFieldsSet
+        If Fields.ListExists Then
+            For Each f As KeyValuePair(Of String, String) In Fields
+                If f.Key = Name_LatestPage Then LatestPage = f.Value
+            Next
+        End If
+    End Sub
+    Public Function XmlFieldsGet() As List(Of KeyValuePair(Of String, String)) Implements IPluginContentProvider.XmlFieldsGet
+        Return New List(Of KeyValuePair(Of String, String)) From {New KeyValuePair(Of String, String)(Name_LatestPage, LatestPage)}
+    End Function
+#End Region
+    Private Property LatestPage As String = String.Empty
+    Private Property Responser As Response = Nothing
+    Private Enum Mode : Internal : External : End Enum
+    Public Sub GetMedia() Implements IPluginContentProvider.GetMedia
+        Try
+            If Not Responser Is Nothing Then Responser.Dispose()
+            Responser = New Response
+            With Responser : .Copy(Settings.Responser) : .Error = EDP.ThrowException : End With
+
+            Dim NextPage$
+            Dim r$
+            Dim _LPage As Func(Of String) = Function() If(LatestPage.IsEmptyString, String.Empty, $"page-{LatestPage}")
+
+            Do
+                r = Responser.GetResponse($"https://www.lpsg.com/threads/{Name}/{_LPage.Invoke}")
+                UserExists = True
+                UserSuspended = False
+                Thrower.ThrowAny()
+                If Not r.IsEmptyString Then
+                    NextPage = RegexReplace(r, NextPageRegex)
+                    UpdateMediaList(RegexReplace(r, PhotoRegEx), Mode.Internal)
+                    UpdateMediaList(RegexReplace(r, PhotoRegExExt), Mode.External)
+                    If NextPage = LatestPage Or NextPage.IsEmptyString Then Exit Do Else LatestPage = NextPage
+                Else
+                    Exit Do
+                End If
+            Loop
+
+            If TempMediaList.ListExists And ExistingContentList.ListExists Then _
+               TempMediaList.RemoveAll(Function(m) ExistingContentList.Exists(Function(mm) mm.URL = m.URL))
+        Catch oex As OperationCanceledException
+        Catch dex As ObjectDisposedException
+        Catch ex As Exception
+            If Responser.StatusCode = Net.HttpStatusCode.ServiceUnavailable Then
+                LogProvider.Add("LPSG not available")
+            Else
+                LogProvider.Add(ex, $"[LPSG.UserData.GetMedia({Name})]")
+            End If
+        End Try
+    End Sub
+    Private Sub UpdateMediaList(ByVal l As List(Of String), ByVal m As Mode)
+        If l.ListExists Then
+            Dim f As SFile
+            Dim u$
+            Dim exists As Boolean
+            Dim r As RParams
+            Dim ude As New ErrorsDescriber(EDP.ReturnValue)
+            For Each url$ In l
+                If Not url.IsEmptyString Then u = SymbolsConverter.Decode(url, {Converters.HTML, Converters.ASCII}, ude) Else u = String.Empty
+                If Not u.IsEmptyString Then
+                    exists = Not IsEmptyString(RegexReplace(u, FileExistsRegEx))
+                    If m = Mode.Internal Then
+                        r = FileRegEx
+                    Else
+                        r = FileRegExExt
+                        If Not exists Then
+                            r = FileRegExExt2
+                            exists = Not IsEmptyString(RegexReplace(u, FileRegExExt2))
+                        End If
+                    End If
+                    If exists Then
+                        f = CStr(RegexReplace(u, r))
+                        f.Path = DataPath.CSFilePSN
+                        f.Separator = "\"
+                        If f.Extension.IsEmptyString Then f.Extension = "jpg"
+                        TempMediaList.ListAddValue(New PluginUserMedia With {.ContentType = UTypes.Picture, .URL = url, .File = f}, TempListAddParams)
+                    End If
+                End If
+            Next
+        End If
+    End Sub
+    Public Sub Download() Implements IPluginContentProvider.Download
+        Try
+            With Responser : .UseWebClient = True : .UseWebClientCookies = True : .ResetError() : End With
+            If TempMediaList.ListExists Then
+                Dim m As PluginUserMedia
+                Dim eweb As ErrorsDescriber = EDP.ThrowException
+                RaiseEvent TotalCountChanged(TempMediaList.Count)
+                For i% = 0 To TempMediaList.Count - 1
+                    Thrower.ThrowAny()
+                    m = TempMediaList(i)
+                    m.DownloadState = UStates.Tried
+                    Try
+                        If Not m.URL.IsEmptyString And Not m.File.IsEmptyString Then
+                            Responser.DownloadFile(m.URL, m.File, eweb)
+                            m.DownloadState = UStates.Downloaded
+                        Else
+                            m.DownloadState = UStates.Skipped
+                        End If
+                    Catch wex As Exception
+                        If Responser.Client.StatusCode = Net.HttpStatusCode.ServiceUnavailable Then
+                            LogProvider.Add("LPSG not available")
+                        Else
+                            m.DownloadState = UStates.Missing
+                            m.Attempts += 1
+                        End If
+                    End Try
+                    RaiseEvent ProgressChanged(1)
+                    TempMediaList(i) = m
+                Next
+            End If
+        Catch oex As OperationCanceledException
+        Catch dex As ObjectDisposedException
+        Catch ex As Exception
+            LogProvider.Add(ex, "[LPSG.UserData.Download]")
+        End Try
+    End Sub
+#Region "IDisposable Support"
+    Private disposedValue As Boolean = False
+    Protected Overridable Overloads Sub Dispose(ByVal disposing As Boolean)
+        If Not disposedValue Then
+            If disposing Then
+                If Not Responser Is Nothing Then Responser.Dispose()
+                If ExistingContentList.ListExists Then ExistingContentList.Clear()
+                If TempPostsList.ListExists Then TempPostsList.Clear()
+                If TempMediaList.ListExists Then TempMediaList.Clear()
+            End If
+            disposedValue = True
+        End If
+    End Sub
+    Protected Overrides Sub Finalize()
+        Dispose(False)
+        MyBase.Finalize()
+    End Sub
+    Public Overloads Sub Dispose() Implements IDisposable.Dispose
+        Dispose(True)
+        GC.SuppressFinalize(Me)
+    End Sub
+#End Region
+End Class

SCrawler.Plugin.XVIDEOS/.editorconfig

@@ -0,0 +1,3 @@
+[*.vb]
+# Modifier preferences
+file_header_template = Copyright (C) 2022  Andy\nThis program is free software: you can redistribute it and/or modify\nit under the terms of the GNU General Public License as published by\nthe Free Software Foundation, either version 3 of the License, or\n(at your option) any later version.\n\nThis program is distributed in the hope that it will be useful,\nbut WITHOUT ANY WARRANTY; without even the implied warranty of\nMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the\nGNU General Public License for more details.\n\nYou should have received a copy of the GNU General Public License\nalong with this program.  If not, see <https://www.gnu.org/licenses/>

SCrawler.Plugin.XVIDEOS/Declarations.vb

@@ -0,0 +1,16 @@
+' Copyright (C) 2022  Andy
+' This program is free software: you can redistribute it and/or modify
+' it under the terms of the GNU General Public License as published by
+' the Free Software Foundation, either version 3 of the License, or
+' (at your option) any later version.
+'
+' This program is distributed in the hope that it will be useful,
+' but WITHOUT ANY WARRANTY
+Imports PersonalUtilities.Functions.RegularExpressions
+Friend Module Declarations
+    Friend ReadOnly Property M3U8Regex As RParams = RParams.DM("http.+?.m3u8.*?(?=')", 0)
+    Friend ReadOnly Property VideoTitleRegex As RParams = RParams.DMS("html5player.setVideoTitle\('(.+)(?='\);)", 1)
+    Friend ReadOnly Property VideoID As RParams = RParams.DMS(".*?www.xvideos.com/(video\d+).*", 1)
+    Friend ReadOnly Property M3U8Reparse As RParams = RParams.DM("NAME=""(\d+).*?""[\r\n]*?(.+)(?=(|[\r\n]+?))", 0, RegexReturn.List)
+    Friend ReadOnly Property M3U8Appender As RParams = RParams.DM("(.+)(?=/.+?\.m3u8.*?)", 0)
+End Module

SCrawler.Plugin.XVIDEOS/M3U8.vb

@@ -0,0 +1,68 @@
+' Copyright (C) 2022  Andy
+' This program is free software: you can redistribute it and/or modify
+' it under the terms of the GNU General Public License as published by
+' the Free Software Foundation, either version 3 of the License, or
+' (at your option) any later version.
+'
+' This program is distributed in the hope that it will be useful,
+' but WITHOUT ANY WARRANTY
+Imports System.Net
+Imports PersonalUtilities.Tools.WEB
+Friend NotInheritable Class M3U8
+    Private Sub New()
+    End Sub
+    Private Shared Function Save(ByVal URLs As List(Of String), ByVal ffmpegFile As SFile, ByVal f As SFile, ByRef Logger As ILogProvider) As SFile
+        Dim CachePath As SFile = Nothing
+        Try
+            If URLs.ListExists Then
+                Dim ConcatFile As SFile = f
+                If ConcatFile.Name.IsEmptyString Then ConcatFile.Name = "PlayListFile"
+                ConcatFile.Extension = "mp4"
+                CachePath = $"{f.PathWithSeparator}_Cache\{SFile.GetDirectories($"{f.PathWithSeparator}_Cache\",,, EDP.ReturnValue).ListIfNothing.Count + 1}\"
+                If CachePath.Exists(SFO.Path) Then
+                    Dim p As New SFileNumbers(ConcatFile.Name,,, New ANumbers With {.Format = ANumbers.Formats.General})
+                    ConcatFile = SFile.Indexed_IndexFile(ConcatFile,, p, EDP.ReturnValue)
+                    Dim i%
+                    Dim eFiles As New List(Of SFile)
+                    Dim dFile As SFile = CachePath
+                    dFile.Extension = "ts"
+                    Using w As New WebClient
+                        For i = 0 To URLs.Count - 1
+                            dFile.Name = $"ConPart_{i}"
+                            w.DownloadFile(URLs(i), dFile)
+                            eFiles.Add(dFile)
+                        Next
+                    End Using
+                    f = FFMPEG.ConcatenateFiles(eFiles, ffmpegFile, ConcatFile, p, EDP.ThrowException)
+                    eFiles.Clear()
+                    Return f
+                End If
+            End If
+            Return Nothing
+        Catch ex As Exception
+            Logger.Add(ex, "[M3U8.Save]")
+            ex.HelpLink = 1
+            Throw ex
+        Finally
+            CachePath.Delete(SFO.Path, SFODelete.None, EDP.None)
+        End Try
+    End Function
+    Friend Shared Function Download(ByVal URL As String, ByVal Appender As String, ByVal ffmpegFile As SFile, ByVal f As SFile, ByRef Logger As ILogProvider) As SFile
+        Try
+            If Not URL.IsEmptyString Then
+                Using w As New WebClient
+                    Dim r$ = w.DownloadString(URL)
+                    If Not r.IsEmptyString Then
+                        Dim l As List(Of String) = ListAddList(Nothing, r.StringFormatLines.StringToList(Of String)(vbNewLine).ListWithRemove(Function(v) v.Trim.StartsWith("#")),
+                                                               New ListAddParams With {.Converter = Function(Input) $"{Appender}/{Input.ToString.Trim}"})
+                        If l.ListExists Then Return Save(l, ffmpegFile, f, Logger)
+                    End If
+                End Using
+            End If
+            Return Nothing
+        Catch ex As Exception
+            If Not ex.HelpLink = 1 Then Logger.Add(ex, $"[M3U8.Download({URL}, {Appender}, {ffmpegFile}, {f})]")
+            Throw ex
+        End Try
+    End Function
+End Class