Compare commits

..

30 Commits

Author SHA1 Message Date
Andy
a5fa935e76 3.0.0.7
Added script usage
Fixed  downloading of LPSG images
Fixed Instagram Stories
Fixed date/time file pattern
2022-04-14 18:12:01 +03:00
Andy
c90dd5637e Update Plugins.md 2022-04-12 05:41:03 +03:00
Andy
9a301ebc5e 3.0.0.6
Added GoTo Start/End channels buttons
Fixed saved Reddit posts downloading
Fixed Reddit accessibility check
Disabled main progress bar progress when downloading saved posts
Added Date and Time for Stories and Tagged Photos
2022-04-04 03:00:22 +03:00
Andy
11a590f14e 3.0.0.5
Added options for Reddit channel and user
Fixed minor bugs and typos
2022-04-02 04:36:38 +03:00
Andy
975d46715c 3.0.0.4
Changed XVIDEOS.M3U8 errors
Added TryCatch to XVIDEOS.UserData.Download
Removed old declarations in UserDataBase
Changed description replacement in UserDataHost
2022-03-26 20:00:55 +03:00
Andy
726fc486ce 3.0.0.3
Added additional 'download all' options
Fixed ListImagesLoader (User.FitToAddParams)
Fixed Instagram default value of SleepTimerOnPostsLimit
Fixed XVIDEOS typo
2022-03-24 19:14:58 +03:00
Andy
ede81f9d05 Update HowToSupport.md 2022-03-23 00:41:18 +03:00
Andy
fccb786e55 3.0.0.2
Added plugins: LPSG, XVIDEOS
Updated hosts
Deleted IResponserContainer
Updated SiteSettingsForm (properties, responser, button, cookies, offset)
Updated plugin dependencies
Fixed Instagram algo typo
2022-03-22 20:14:17 +03:00
Andy
2be52c7c56 Update README.md
Added additional authorization info
2022-03-20 16:00:10 +03:00
Andy
dd7f071e1a 3.0.0.1
Added DownloadLimitDate, update and Reset functions in the plugin, DateLimit property in the IPluginContentProvider
Fixed donation button link, images loading error, cookies error, instagram settings form, minor bugs.
2022-03-20 02:17:53 +03:00
Andy
617a7f702b Update ProgramsComparison.md
Update ProgramsComparison
2022-03-18 03:48:06 +03:00
Andy
05c84c2c08 3.0.0.0
Parsing profiles descriptions (Reddit and Twitter) and updating it
Filters: deleted, suspended, dates
Collections containing deleted profiles are marked in blue
Marked collection context elements
Find profile in the main window from the info form
New hotkeys in the info form: up, down, find, enter
New hotkey in the main window: enter
New list refill algo
Added copying user pictures from all channels
Changed view modes
Changed comparer and ToString of UserDataBase
New parameter added to channels stats (my users)
Added view mode "details"
Fixed twitter files overriding
Fixed full parsing of reddit posts
Fixed Insta timers and minors
Fixed library fatal
Removed UserDataBind comparer override
Added GetUserMediaOnly for reddit users from channels
Added Reddit availability check with DownDetector
Added PLUGINS
2022-03-17 21:15:22 +03:00
Andy
19373ec4ba Create FAQ.md 2022-02-10 11:09:28 +03:00
Andy
265f62890b Update issue templates 2022-02-07 12:16:09 +03:00
Andy
9ba3df8afd Update README.md 2022-02-07 10:54:31 +03:00
Andy
5b64e5563b 2.0.0.4
Fixed network paths bug
Updated classes for new library structures and functions
Fixed some typos
Fixed minor bugs
SitePaths
2022-02-07 09:43:35 +03:00
Andy
1bc048578d Update issue templates 2022-02-06 08:34:45 +03:00
Andy
5d5ce29f36 2.0.0.3
Added GetUserMediaOnly for Reddit
Fixed Reddit CrossPosts issue
Fixed validating new users (collections)
Fixed incorrect count of Instagram download tasks
2022-02-02 07:34:36 +03:00
Andy
2a2fc45a91 Create FUNDING.yml 2022-02-02 03:56:11 +03:00
Andy
b17e6ac40a Fix typo 2022-02-01 07:02:26 +03:00
Andy
301f2a6b03 Update CONTRIBUTING.md 2022-02-01 07:00:51 +03:00
Andy
fd72ba0e2a Update README.md 2022-02-01 07:00:28 +03:00
Andy
9146b16cfa Update README.md 2022-02-01 06:55:19 +03:00
Andy
523794d210 Update README.md 2022-01-23 07:08:40 +03:00
Andy
f1ba2ecd77 2.0.0.2
Site settings have been expanded, some functions and dependencies have been changed.
Removed unused elements in UserDataBase, added additional xml fields, added error executor.
Created a basic download function.
Added Instagram saved posts and 429 bypass.
Added channel statistics.
Added site redgifs.
Updated sites algorithms.
Other improvements.
Updated downloader algorithm.
2022-01-23 05:34:09 +03:00
Andy
7da1ccf1ae Update README.md 2022-01-14 17:17:21 +03:00
Andy
751005284d Create CONTRIBUTING.md 2022-01-14 17:13:27 +03:00
Andy
aa824138e2 Update README.md 2022-01-04 01:21:15 +03:00
Andy
0cacba4144 2.0.0.1
Added download individual imgur media
Fixed missing file
Updated global settings form
Fixed incorrect filling user params
2021-12-29 09:55:29 +03:00
Andy
69ded7b400 Update README.md 2021-12-27 21:48:41 +03:00
182 changed files with 17053 additions and 3669 deletions

13
.github/FUNDING.yml vendored Normal file
View File

@@ -0,0 +1,13 @@
# These are supported funding model platforms
github: # Replace with up to 4 GitHub Sponsors-enabled usernames e.g., [user1, user2]
patreon: # Replace with a single Patreon username
open_collective: # Replace with a single Open Collective username
ko_fi: andyprogram
tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
liberapay: # Replace with a single Liberapay username
issuehunt: # Replace with a single IssueHunt username
otechie: # Replace with a single Otechie username
lfx_crowdfunding: # Replace with a single LFX Crowdfunding project-name e.g., cloud-foundry
custom: ['https://blockchair.com/bitcoin/address/BC1Q0NH839FT5TA44DD7L7RLR97XDQAG9V8D6N7XET']

35
.github/ISSUE_TEMPLATE/bug_report.md vendored Normal file
View File

@@ -0,0 +1,35 @@
---
name: Bug report
about: Create a report to help us improve
title: "[BUG]"
labels: bug
assignees: ''
---
**Describe the bug**
A clear and concise description of what the bug is.
**To Reproduce**
Steps to reproduce the behavior:
1. Profile URL:
2. Do something
3. See error
**Log data**
If the program log contains any data.
**Expected behavior**
A clear and concise description of what you expected to happen.
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Release information (please complete the following information):**
- OS [e.g. Windows 10, Windows 11]
- Architecture [e.g. x86, x64]
- Version [e.g. 2.0.0.0]
- NET.Framework version
**Additional context**
Add any other context about the problem here.

10
.github/ISSUE_TEMPLATE/custom.md vendored Normal file
View File

@@ -0,0 +1,10 @@
---
name: Custom issue template
about: Describe this issue template's purpose here.
title: ''
labels: ''
assignees: ''
---

View File

@@ -0,0 +1,20 @@
---
name: Feature request
about: Suggest an idea for this project
title: "[REQUEST]"
labels: enhancement
assignees: ''
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.

12
.github/ISSUE_TEMPLATE/plugin_add.md vendored Normal file
View File

@@ -0,0 +1,12 @@
---
name: Add plugin
about: Add plugin to plugin list
title: "[NEW PLUGIN]"
labels: 'New Plugin'
assignees: ''
---
Plugin address:
Plugin name:
Plugin site:

41
CONTRIBUTING.md Normal file
View File

@@ -0,0 +1,41 @@
# Contributor's Guide
I welcome requests! Follow these steps to contribute:
1. Find an [issue](https://github.com/AAndyProgram/SCrawler/issues) that needs assistance.
2. Let me know you are working on it by posting a comment on the issue.
3. If you find an error in the code, please provide a link to the file and the line number.
4. If you have a suggestion to change the code, you can post a block of code to replace. I don't currently have time to learn pull requests, so it might work this way.
# How to build from source
1. Delete the "PersonalUtilities" project from the solution.
2. Add the latest version of the "PersonalUtilities.dll" library (from the [latest release](https://github.com/AAndyProgram/SCrawler/releases/latest)).
3. Import PersonalUtilities.Functions for the whole project.
**Always use the correct "PersonalUtilities.dll" library. You must download this library from the release of the code you downloaded.**
# How to request a new site
1. Check [issues](https://github.com/AAndyProgram/SCrawler/issues) (open and [closed](https://github.com/AAndyProgram/SCrawler/issues?q=is%3Aissue+is%3Aclosed)) and [discussions](https://github.com/AAndyProgram/SCrawler/discussions) to find your issue. Perhaps I have already answered your request.
2. If you don't find anything, create a new issue with your request. I usually reply as soon as possible (within the next few hours).
- If I'm interested in a site you want to add, it may be added in future releases.
- If the site has an API that does not require authorization, it may be added in the coming releases.
- You can make it faster by posting a link to the API. **I don't use OAuth authentication** in my application, so if it's not too hard to make a new parsing algorithm **without OAuth** authorization, I can start developing it in the coming days. Otherwise, I need time to figure out how to do it.
- If the site does not have an API that does not require authorization, this may take some time.
- If you will be posting request urls **without OAuth** authentication, I might consider adding your site if I have time.
- If I'm **not** interested in the site you want to add, you can pay to have it added by making a donation of approximately $10. **But before that, you still need to create an issue. If I'm not interested, you can offer me a deal to develop it for money. I'll check the site you want to add, check the availability of the API and tell you how much time I need to develop it and the price. If you agree, I will do it.** [![ko-fi](https://www.ko-fi.com/img/githubbutton_sm.svg)](https://ko-fi.com/andyprogram)
# Sites I will never develop
- Facebook
# Sites requested by users
- TikTok
- API for receiving data without authorization was not found. Therefore, I don't have time to start developing this site parsing algorithm. If anyone knows of requests that may collect data without OAuth authentication, please let me know.
# Contact me
[![matrix](https://img.shields.io/badge/Matrix-%40andyprogram%3Amatrix.org-informational)](https://matrix.to/#/@andyprogram:matrix.org)

View File

@@ -1,3 +1,168 @@
# 3.0.0.7
- Added
- Ability to run a script after the user download if complete
- Hotkey ```F2``` for additional options in the user creation form
- Fixed
- (Issue #32) In some cases, Date and Time are still not added for Stories and Tagged Photos
- (Issue #33) Instagram Stories downloading error
- LPSG downloader does not download all content
# 3.0.0.6
- Added
- ```GoTo Start``` channels button
- ```GoTo End``` channels button
- Fixed
- In some cases, saved Reddit posts didn't fully download
- Incorrect Reddit accessibility check algorithm
- Incorrect behavior of the main progress bar when downloading saved posts
- (Issue #25) Date and Time not added for Stories and Tagged Photos
# 3.0.0.5
- Added
- ```New```, ```Hot```, ```Top``` Reddit channel and user download modes
# 3.0.0.4
- Fixed
- External plugins do not save information about downloaded files
- The user cannot be added to the collection if a special path has been specified.
# 3.0.0.3
- Added
- Download all by specific sites
- Download all, ignoring the ```Ready for download``` option
- Download all by specific sites, ignoring the ```Ready for download``` option
- Fixed
- (Issue #19) Typo in default Instagram settings (Post limit timer)
- Typo when applying "Download UHD" in XVIDEOS plugin
- The sites filter does not work unless the "Fast profiles loading" option is enabled.
# 3.0.0.2
- Added
- **LPSG** site plugin
- **XVIDEOS** site plugin
- Updated
- Plugin provider
- Fixed
- Minor bugs
# 3.0.0.1
- Added
- Download data up to a specific date
- Update and Reset functions in the plugin (ISiteSettings)
- PostsDateLimit propperty in the plugin (IPluginContentProvider)
- Fixed
- The donation button redirects to a broken link
- In some cases, an error occurs when fast loading images
- In some cases, cookies are not saved
- Some design fixes
- Minor bugs
# 3.0.0.0
**Attention! This version of the program makes changes user data file (Users.xml). Once you start using this version, you will not be able to use previous versions of the program. Therefore, it is highly recommended to archive the program settings folder and archive the users' data files (you can use the [```ArchiveSCrawlerUsersDataFiles.bat```](Tools/ArchiveSCrawlerUsersDataFiles.bat) tool to archive the data files of all users).**
- Added
- **PLUGINS SUPPORT**
- **Gfycat** site support
- Description of Twitter and Reddit user profiles
- Filter users by profile status "Suspended"
- Filter users by profile status "Deleted"
- Filter profiles that haven't downloaded new data since specific date
- Collections that contain non-existent profiles will be marked in blue
- Ability to find and activate a user in the main window from the Info form
- Ability to copy user images from all channels you have when adding a user from a channel
- Reddit default option "Get user media only" if now also used when creating new users from channels
- Ability to update user description every time
- ```Enter``` hotkey in the download info form to open the user's folder
- ```Enter``` hotkey in the main window to open the user's folder
- Channel statistics are supplemented by "existing users"
- ```Up``` and ```Down``` navigation buttons in the Info form
- ```Find``` button on the Info form to find the user in the main window
- "Details" view mode
- Fast loading of profiles in the main window. **Be careful with this setting. Fast loading leads to the highest CPU usage.**
- Reddit availability check with DownDetector
- Ability to [open folders with a specific program](https://github.com/AAndyProgram/SCrawler/wiki/Settings#folder-command)
- (Request #16) Ability to remove a user from the collection without deletion
- (Request #17) **Instagram Tagged** photos downloading
- (Request #17) **Instagram Stories** downloading
- Deleting data to recycle bin
- Updated
- "List" view mode
- Fixed
- Twitter reloads existing media
- Reddit saved posts downloader downloads all posts every time
- Minor bug that caused Instagram tasks timers to run longer
- A library error that in some cases leads to a fatal program error
- (Issue #16) Cannot delete a user that is in the collection.
At the requests of some users, I added [screenshots](ProgramScreenshots) of the program and added screenshots to [ReadMe](README.md) and the [guide](https://github.com/AAndyProgram/SCrawler/wiki).
# 2.0.0.4
**Removed compatibility of program settings with version 1.0.0.4 and lower.**
**If your program version is 1.0.0.4 and lower, it is strongly recommended that you upgrade to release 2.0.0.1 to update the program settings (and run the program). Then update to this release. Otherwise, you will have to configure the program settings again**
**If your program version is 1.0.1.0 or higher, you should not pay attention to this message.**
- Added
- Ability to specify the path to store saved posts
- Fixed
- **Error when specifying network paths**
- Minor bugs
# 2.0.0.3
**Removed compatibility of program settings with version 1.0.0.4 and lower.**
**If your program version is 1.0.0.4 and lower, it is strongly recommended that you upgrade to release 2.0.0.1 to update the program settings (and run the program). Then update to this release. Otherwise, you will have to configure the program settings again**
**If your program version is 1.0.1.0 or higher, you should not pay attention to this message.**
- Added
- The "Get User Media Only" setting is now available for Reddit. If checked then "CrossPosts" will be skipped, otherwise "CrossPosts" will be included.
- Fixed
- In some cases, the program did not parse all Reddit posts.
- Collection ignored when validated when creating a new user
- Incorrect number of Instagram profiles downloads per session
# 2.0.0.2
**This is the last release that supports program settings of version 1.0.0.4 and lower. Compatibility of program settings with version 1.0.0.4 and lower will be removed in future releases. It is strongly recommended that you upgrade to this release before future releases. Otherwise, you will have to configure the program settings again. If your program version is 1.0.1.0 or higher, you should not pay attention to this message.**
- Added
- Tray icon
- Close program to tray
- Close confirmation dialog
- **Separated thread for downloading Instagram profiles**
- **Wait timers to bypass Instagram error "Too Many Requests" (429)**
- **Downloading saved Instagram posts** *(requires a second InstaHash)*
- Downloading saved posts (from Reddit and Instagram) form
- Tray notification when download is complete (Instagram notification separate from other)
- Downloading not downloaded Instagram posts when a 429 error is encountered and/or the user stops downloading
- Separate progress bar for downloading Instagram profiles
- Clear information about downloaded profiles of the current session in the "Download info form"
- Increased the number of Instagram posts (from 12 to 50) received per request
- Channels' statistics
- **RedGifs profiles support**
- Fixed
- The program was showing incorrect information about the total numbers of images and videos downloaded when a Reddit user was created from a channel
# 2.0.0.1
- Added
- Download individual Imgur media files (use the "Download video" form).
- Fixed
- Incorrect filling of user parameters in the user creation form
- In some cases, the global settings cannot be saved.
# 2.0.0.0 # 2.0.0.0
- Added - Added
@@ -6,7 +171,7 @@
- Group for regular channels in the main window - Group for regular channels in the main window
- Ability to change user/collection path - Ability to change user/collection path
- Imgur albums downloading - Imgur albums downloading
- NSFW Imgur content bypass (requires 'CilentID') - NSFW Imgur content bypass (requires 'ClientID')
- Special user folder - Special user folder
- Remove user while keeping data - Remove user while keeping data
- Disabled overriding user preferences when creating a new user if it already exists in the destination (in case of deleting a user with saving data). - Disabled overriding user preferences when creating a new user if it already exists in the destination (in case of deleting a user with saving data).

99
FAQ.md Normal file
View File

@@ -0,0 +1,99 @@
# Frequently asked questions
**Please read the [GUIDE](https://github.com/AAndyProgram/SCrawler/wiki/) Before asking a question!**
**Also read [here](README.md) for basic information.**
Most of your questions are already answered. All settings, functions, buttons and everything else described in the guide.
Any other questions I will keep in this file.
----
#### Q: **HOW TO SETUP COOKIES**
A: https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-set-up-cookies
----
#### Q: **I can't copy cookies.**
A: Use the mouse. Don't use ```Ctrl``` + ```A```!
----
#### Q: **Does this program have GUI or CLI.**
A: This is a GUI program.
----
#### Q: **Will CLI be added in the future?**
A: I do not think so.
----
#### Q: **I want to add "...." site. How to request.**
A: How to request a new site you can read [here](CONTRIBUTING.md#how-to-request-a-new-site)
----
#### Q: **Twitter/Instagram download failed.**
A: Check your credentials. Both of these sites require cookies. Check your [Twitter tokens](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-find-twitter-tokens) and [Instagram settings](https://github.com/AAndyProgram/SCrawler/wiki/Settings#instagram-settings). If all settings are set, but nothing works, go to [create a new issue](https://github.com/AAndyProgram/SCrawler/issues). Don't forget to attach the LOG.
----
#### Q: **I have set credentials but still nothing is downloading**
A: Click the ```Start downloading``` button
----
#### Q: **Where can I find the release?**
A: https://github.com/AAndyProgram/SCrawler/releases/latest
----
#### Q: **How to run the program?**
A: Double-click ```SCrawler.exe```
----
#### Q: **Where to find binaries?**
A: https://github.com/AAndyProgram/SCrawler/releases/latest
----
#### Q: **Does the program remember the last download and check for new posts, downloading only new posts? Or does the program download the entire profile every time?**
A: The program stored posts IDs in users' folders. For the first time, the program downloads the entire profile. All subsequent times the program will check for new posts and download **only new posts**!
----
#### Q: **How to redownload all data**
A: Double-click on the user you want to redownload. In the opened window open folder setting. Delete the files ending with ```_Data.xml``` and ```_Posts.txt```. Download this user again.
----
#### Q: **How to remove the label**
A: There is no functionality to remove an individual label. You can open the ```Labels.txt``` file in the program settings folder and delete any label you want. You also can delete this file (```Labels.txt```). In this case, when the program starts, the list of labels list will be updated with only existing labels (from the user data files).
----
#### Q: **How to remove a user from the blacklist**
A: Just add that user back to the program. In the dialog box that opens, click on the ```Add and remove from blacklist``` button.
----
#### Q: **Can you add a step-by-step guide or video on how to use the program?**
A: **NO**! I will not do it. If you want, you can create a video tutorial and send it to me. Then I add it. All options and what each option does described on the wiki. The wiki also contains a description of all settings and how-to configure them. For complex settings, there is a steep-by-steep guide. Read the [main](README.md) information and [GUIDE](https://github.com/AAndyProgram/SCrawler/wiki/) and you won't have any problems. I have developed a program with an intuitive interface. There is a Settings button, download buttons, a context menu that drops down when a user is clicked, and other controls. Anyone can use it.

11
HowToSupport.md Normal file
View File

@@ -0,0 +1,11 @@
Your support is very valuable to me. Any support is greatly appreciated. Your support encourages me to make new features, update the program, add new sites, etc.
You can support the program by:
- **Bitcoin**: bitcoin:BC1Q0NH839FT5TA44DD7L7RLR97XDQAG9V8D6N7XET
- :heavy_dollar_sign: make a donation on this site: https://ko-fi.com/andyprogram
- :repeat: make a post about my program on your profile (Reddit, Twitter, Instagram and any other social networks)
- :speech_balloon: tell your friends about the program
- :heart: like the program on this site: https://alternativeto.net/software/scrawler/about/
- suggest my program as an alternative ([on this site](https://alternativeto.net/software/scrawler/about/)) to any program you have used before
I would be very grateful for any support! :blush:

6
Plugins.md Normal file
View File

@@ -0,0 +1,6 @@
List of available plugins:
- LPSG
- XVIDEOS
Tools:
- [image2post](https://github.com/unknown81311/SCrawler-image2post) by @unknown81311: **get reddit post URL from file.**

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.3 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 10 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.9 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 17 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.5 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.2 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 369 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 377 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.9 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 10 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 26 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 23 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 14 KiB

78
ProgramsComparison.md Normal file
View File

@@ -0,0 +1,78 @@
# 4K Stogram
https://www.4kdownload.com/products/product-stogram
| Option | SCrawler | 4K Stogram |
| ---- | ---- | ---- |
| User managament | **Advanced** | Primitive |
| Labeling users | **Yes** | No |
| Filtering | **Yes** | No |
| Collections | **Yes** | No |
| Specific user folders | **Yes** | No |
| Favorite / Temporary user options | **Yes** | No |
| Plugins support | **Yes** | No |
| Download posts by username | Yes | Yes |
| Download posts by hashtag | No | **Yes** |
| Download posts by location | No | **Yes** |
| Save Private Instagram Content with Permission| Yes | Yes |
| Download Instagram Stories and Highlights | Yes | Yes |
| See Others Instagram Feed As Your Own | No | **Yes** |
| Download Instagram Video Posts | Yes | Yes |
| Backup Your Instagram Account | Yes | Yes |
| Save Instagram Posts by Date | No (only limited download) | **Yes** |
| Download Instagram Saved Posts | Yes | Yes |
| Download Instagram Tagged Posts | Yes | Yes |
| Export and import subscriptions | No | **Yes** |
| **Paid** | **No** | Yes |
| **Free options** | **The program is completely free** | Only **ONE** profile downloading and only **200 posts** per day |
| Permitted Commercial Use | **Yes** | Starting from 43.56 EUR |
| Automatic Subscriptions Update | **Free** | Paid (43.56 EUR) |
| Posts and Captions Export | No | Paid (43.56 EUR) |
| Advertisements free | **No ADs at all for free** | Paid (14.52) |
| Operating Systems | Windows 7+ | Windows 7+, MacOS 10.13+, Ubuntu x64 |
| Select want content type to download | **Yes** | No |
| Instagram support | Yes | Yes |
| Twitter support | **Yes** | No |
| Reddit support | **Yes** | No |
| Other sites support | **Yes** | No |
| Still supported | Yes | Yes |
# RipMeApp
https://github.com/RipMeApp/ripme
| Option | SCrawler | RipMeApp |
| ---- | ---- | ---- |
| User managament | **Advanced** | No |
| Labeling users | **Yes** | No |
| Filtering | **Yes** | No |
| Collections | **Yes** | No |
| Specific user folders | **Yes** | No |
| Favorite / Temporary user options | **Yes** | No |
| Plugins support | **Yes** | No |
| Download posts by username | Yes | Yes |
| Download posts by hashtag | No | No |
| Download posts by location | No | No |
| Save Private Instagram Content with Permission| Yes | Yes |
| Download Instagram Stories | Yes | Yes |
| Download Instagram Video Posts | Yes | Yes |
| Backup Your Instagram Account | Yes | Yes |
| Download Instagram Saved Posts | **Yes** | No |
| Download Instagram Tagged Posts | **Yes** | No |
| Export and import subscriptions | No | No |
| **Paid** | **No** | **No** |
| **Free options** | The program is completely free | The program is completely free, but site limits are not declared |
| Operating Systems | Windows 7+ | Windows, MacOS, Linux |
| Select want content type to download | Yes | Yes |
| Suported sites | 3 internal and any site using plugins | 86+ sites (declared) |
| Other sites support | **Yes** | No |
| Still supported | **Yes** | **No (last release date May 4, 2021)** |
# gallery-dl
https://github.com/mikf/gallery-dl
**CLI tool**! Configured with JSON files only. Users need to learn complex configuration options, JSON, commands to use that tool. Very difficult to configure.
SCrawler has advanced user management, collections, labels, groups, a beautiful view, GUI, the ability to add plugins for other sites and much more. Just try it and compare.

105
README.md
View File

@@ -1,62 +1,129 @@
# Social networks crawler # Social networks crawler
Program for downloading photo and video from Reddit, Twitter and Instagram [![GitHub release (latest by date)](https://img.shields.io/github/v/release/AAndyProgram/SCrawler)](https://github.com/AAndyProgram/SCrawler/releases/latest)
[![GitHub](https://img.shields.io/github/license/AAndyProgram/SCrawler)](https://github.com/AAndyProgram/SCrawler/)
[![FAQ](https://img.shields.io/badge/FAQ-green)](FAQ.md)
[![GUIDE](https://img.shields.io/badge/GUIDE-green)](https://github.com/AAndyProgram/SCrawler/wiki)
[![How to support](https://img.shields.io/badge/HowToSupport-green)](HowToSupport.md)
Do you like this program? Consider adding to my coffee fund by making a donation to show your support. :) A program to download photo and video from [any site](#supported-sites) (e.g. Reddit, Twitter, Instagram).
Do you like this program? Consider adding to my coffee fund by making a donation to show your support. :blush:
[![ko-fi](https://www.ko-fi.com/img/githubbutton_sm.svg)](https://ko-fi.com/andyprogram) [![ko-fi](https://www.ko-fi.com/img/githubbutton_sm.svg)](https://ko-fi.com/andyprogram)
**Bitcoin**: bitcoin:BC1Q0NH839FT5TA44DD7L7RLR97XDQAG9V8D6N7XET
![Main window](ProgramScreenshots/MainWindow.png)
![Channels window](ProgramScreenshots/Channels.png)
# What can program do: # What can program do:
- Download pictures and videos from users' profiles and subreddits: - Download pictures and videos from users' profiles and subreddits:
- Reddit images; - Reddit images;
- Reddit galleries of images; - Reddit galleries of images;
- Redgifs hosted videos (https://www.redgifs.com/); - Redgifs hosted videos (https://www.redgifs.com/);
- Reddit hosted videos (downloading Reddit hosted video is going through ffmpeg); - Reddit hosted videos (downloading Reddit hosted video is going through ffmpeg (**ffmpeg only works with the x64 program**));
- Twitter images and videos; - Twitter images and videos;
- Instagram images and videos. - Instagram images and videos.
- Imgur images, galleries and videos
- Gfycat videos
- [Other](#supported-sites) supported sites
- Parse [channel and view data](https://github.com/AAndyProgram/SCrawler/wiki/Channels). - Parse [channel and view data](https://github.com/AAndyProgram/SCrawler/wiki/Channels).
- Download [saved Reddit posts](https://github.com/AAndyProgram/SCrawler/wiki/Home#saved-posts). - Download [saved Reddit and Instagram posts](https://github.com/AAndyProgram/SCrawler/wiki/Home#saved-posts).
- Add users from parsed channel. - Add users from parsed channel.
- **Advanced user management.**
- Labeling users. - Labeling users.
- Adding users to favorites and temporary.
- Filter exists users by label or group. - Filter exists users by label or group.
- Selection of media types you want to download (images only, videos only, both) - Selection of media types you want to download (images only, videos only, both)
- Download a special video, image or gallery
- Making collections (grouping users into collections)
- Specifying a user folder (for downloading data to another location)
- Changing user icons
- Changing view modes
- ...and many others...
# Supported sites
- **Reddit**
- **Twitter**
- **Instagram**
- RedGifs
- Imgur
- Gfycat
- LPSG
- XVIDEOS
- [Other sites](Plugins.md)
# How does it works: # How does it works:
First, the program downloads the full profile. After the program downloads only new posts. The program remembers downloaded posts.
## Reddit ## Reddit
The program parses all user posts, obtain MD5 images hash and compares them with existing ones to remove duplicates. Then the media will be downloaded. The program parses all user posts, obtain MD5 images hash and compares them with existing ones to remove duplicates. Then the media will be downloaded.
## Twitter and Instagram ## Other sites
The program parses all user posts and compares file names with existing ones to remove duplicates. Then the media will be downloaded. The program parses all user posts and compares file names with existing ones to remove duplicates. Then the media will be downloaded.
You can read about Instagram restrictions [here](https://github.com/AAndyProgram/SCrawler/wiki/Settings#instagram-limits) You can read about Instagram restrictions [here](https://github.com/AAndyProgram/SCrawler/wiki/Settings#instagram-limits)
# Requirements: ## How to request a new site
- Windows 7, 8, 9, 10, 11 with NET Framework 4.6.1 or higher Read [here](CONTRIBUTING.md#how-to-request-a-new-site) about
- Authorization cookies and tokens for Twitter (if you want to download data from Twitter)
- Authorization cookies Instagram (if you want to download data from Instagram) # Requirements
- ffmpeg library for downloading videos hosted on Reddit (you can download it from the [official repo](https://github.com/GyanD/codexffmpeg/releases/tag/2021-01-12-git-ca21cb1e36) or [from my first release](https://github.com/AAndyProgram/SCrawler/releases/download/1.0.0.0/ffmpeg.zip))
- Don't put program in the ```Program Files``` system folder (this is portable program and program settings are stored in the program folder) - Windows 7, 8, 9, 10, 11 with NET Framework 4.6.1 or higher (v4.6.1 must be installed). You can check version compatibility with this [tool](Tools/NET.FrameworkVersion.ps1).
- Just unzip the program archive to any folder, copy the file ```ffmpeg.exe``` into it and enjoy. :) - Authorization [cookies](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-set-up-cookies) and [tokens](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-find-twitter-tokens) for Twitter (if you want to download data from Twitter)
- Authorization [cookies](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-set-up-cookies) and [Hash](https://github.com/AAndyProgram/SCrawler/wiki/Settings#instagram) for Instagram (if you want to download data from Instagram), [Hash 2](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-find-instagram-hash-2) for saved Instagram posts, Instagram [stories authorization headers](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-find-instagram-stories-authorization-headers) for Stories and Tagged data
- ffmpeg library for downloading videos hosted on Reddit (you can download it from the [official repo](https://github.com/GyanD/codexffmpeg/releases/tag/2021-01-12-git-ca21cb1e36) or [from my first release](https://github.com/AAndyProgram/SCrawler/releases/download/1.0.0.0/ffmpeg.zip)). **ffmpeg only works with the x64 version of the program.**
# Guide
**Full guide you can find [here](https://github.com/AAndyProgram/SCrawler/wiki)**
# Installation
**Just unzip the program archive to any folder, copy the file ```ffmpeg.exe``` into it and enjoy.** :blush:
**Don't put program in the ```Program Files``` system folder (this is portable program and program settings are stored in the program folder)**
# Updating # Updating
Just download [latest](https://github.com/AAndyProgram/SCrawler/releases/latest) version and unpack it into the program folder. Before starting a new version, I recommend making a backup copy of the program settings folder. Just download [latest](https://github.com/AAndyProgram/SCrawler/releases/latest) version and unpack it into the program folder. **Before starting a new version, I recommend making a backup copy of the program settings folder.**
# How to build from source
1. Delete the "PersonalUtilities" project from the solution.
1. Add the latest version of the "PersonalUtilities.dll" library (from the [latest release](https://github.com/AAndyProgram/SCrawler/releases/latest)).
1. Import PersonalUtilities.Functions for the whole project.
# How to make a plugin
Read about how to make plugin [here](https://github.com/AAndyProgram/SCrawler/wiki/Plugins).
# How to support
Read more about how to support the program [here](HowToSupport.md).
# Settings and usage # Settings and usage
The program has an intuitive interface. The program has an intuitive interface.
Just add a user profile and click the ```Start downloading``` button. You need to set up authorization for Twitter and Instagram:
- Authorization [cookies](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-set-up-cookies) and [tokens](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-find-twitter-tokens) for **Twitter** (if you want to download data from Twitter)
- Authorization [cookies](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-set-up-cookies) and [Hash](https://github.com/AAndyProgram/SCrawler/wiki/Settings#instagram) for **Instagram** (if you want to download data from Instagram), [Hash 2](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-find-instagram-hash-2) for **saved Instagram posts**, Instagram [stories authorization headers](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-find-instagram-stories-authorization-headers) for **Stories** and **Tagged data**
Just add a user profile and **click the ```Start downloading``` button**.
You can add users by patterns: You can add users by patterns:
- https://www.instagram.com/SomeUserName - https://www.instagram.com/SomeUserName
- https://twitter.com/SomeUserName - https://twitter.com/SomeUserName
- https://reddit.com/user/SomeUserName - https://reddit.com/user/SomeUserName
- https://reddit.com/r/SomeSubredditName - https://reddit.com/r/SomeSubredditName
- https://www.redgifs.com/users/SomeUserName
- u/SomeUserName - u/SomeUserName
- r/SomeSubredditName - r/SomeSubredditName
- SomeUserName (in this case, you need to select the user's site) - SomeUserName (in this case, you need to select the user's site)
@@ -64,10 +131,16 @@ You can add users by patterns:
Read more about adding users and subreddits [here](https://github.com/AAndyProgram/SCrawler/wiki/Users) Read more about adding users and subreddits [here](https://github.com/AAndyProgram/SCrawler/wiki/Users)
**Full guide you can find [here](https://github.com/AAndyProgram/SCrawler/wiki)** ![Add user](ProgramScreenshots/CreateUserClear.png)
## Using program as just video downloader # Using program as just video downloader
Create a shortcut for the program. Open shortcut properties. In the ```Shortcut``` tab, in the ```Target``` field, just add the letter ```v``` at the end across the space. Create a shortcut for the program. Open shortcut properties. In the ```Shortcut``` tab, in the ```Target``` field, just add the letter ```v``` at the end across the space.
Example: ```D:\Programs\SCrawler\SCrawler.exe v``` Example: ```D:\Programs\SCrawler\SCrawler.exe v```
![Separate video downloader](ProgramScreenshots/SeparateVideoDownloader.png)
# Contact me
[![matrix](https://img.shields.io/badge/Matrix-%40andyprogram%3Amatrix.org-informational)](https://matrix.to/#/@andyprogram:matrix.org)

View File

@@ -0,0 +1,3 @@
[*.vb]
# Modifier preferences
file_header_template = Copyright (C) 2022 Andy\nThis program is free software: you can redistribute it and/or modify\nit under the terms of the GNU General Public License as published by\nthe Free Software Foundation, either version 3 of the License, or\n(at your option) any later version.\n\nThis program is distributed in the hope that it will be useful,\nbut WITHOUT ANY WARRANTY; without even the implied warranty of\nMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the\nGNU General Public License for more details.\n\nYou should have received a copy of the GNU General Public License\nalong with this program. If not, see <https://www.gnu.org/licenses/>

Binary file not shown.

After

Width:  |  Height:  |  Size: 21 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 656 B

View File

@@ -0,0 +1,48 @@
' Copyright (C) 2022 Andy
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
' (at your option) any later version.
'
' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY
Imports PersonalUtilities.Functions.RegularExpressions
Friend Module Declarations
Friend ReadOnly Property PhotoRegEx As RParams = RParams.DM("(https://www.lpsg.com/attachments)(.+?)(?="")", 0, RegexReturn.List)
Friend ReadOnly Property PhotoRegExExt As New RParams("img.data.src=""(/proxy[^""]+?)""", Nothing, 1, RegexReturn.List) With {
.Converter = Function(Input) $"https://www.lpsg.com/{SymbolsConverter.HTML.Decode(Input)}"}
Friend ReadOnly Property NextPageRegex As RParams = RParams.DMS("<link rel=""next"" href=""(.+?/page-(\d+))""", 2)
Private Const FileUrlRegexDefault As String = "([^/]+?)(jpg|jpeg|gif|png|webm)"
Private ReadOnly InputFReplacer As New ErrorsDescriber(EDP.ReturnValue)
Private ReadOnly InputForbidRemover As Func(Of String, String) = Function(Input) If(Input.IsEmptyString, Input, Input.StringRemoveWinForbiddenSymbols(, InputFReplacer))
Friend ReadOnly Property FileRegEx As New RParams(FileUrlRegexDefault, Nothing, 0) With {
.Converter = Function(ByVal Input As String) As String
Input = InputForbidRemover.Invoke(Input)
If Not Input.IsEmptyString Then
Dim lv$ = Input.Split("-").LastOrDefault
If Not lv.IsEmptyString Then
Input = Input.Replace($"-{lv}", String.Empty)
Input &= $".{lv}"
End If
End If
Return Input
End Function}
Friend ReadOnly Property FileRegExExt As New RParams(FileUrlRegexDefault, 0, Nothing, InputForbidRemover)
Friend ReadOnly Property FileRegExExt2 As New RParams("([^/]+?)(?=(\Z|&))", 0, Nothing, InputForbidRemover)
Friend ReadOnly Property FileExistsRegEx As RParams = RParams.DMS(FileUrlRegexDefault, 2)
Private Class PUMComparer : Implements IEqualityComparer, IEqualityComparer(Of PluginUserMedia)
Private Overloads Function Equals(ByVal x As PluginUserMedia, ByVal y As PluginUserMedia) As Boolean Implements IEqualityComparer(Of PluginUserMedia).Equals
Return x.URL = y.URL
End Function
Private Function IEqualityComparer_Equals(ByVal x As Object, ByVal y As Object) As Boolean Implements IEqualityComparer.Equals
Return DirectCast(x, PluginUserMedia).URL = DirectCast(y, PluginUserMedia).URL
End Function
Private Overloads Function GetHashCode(ByVal Obj As Object) As Integer Implements IEqualityComparer.GetHashCode
Throw New NotImplementedException()
End Function
Private Overloads Function GetHashCode(ByVal Obj As PluginUserMedia) As Integer Implements IEqualityComparer(Of PluginUserMedia).GetHashCode
Throw New NotImplementedException()
End Function
End Class
Friend ReadOnly TempListAddParams As New ListAddParams(LAP.NotContainsOnly) With {.Comparer = New PUMComparer}
End Module

View File

@@ -0,0 +1,13 @@
'------------------------------------------------------------------------------
' <auto-generated>
' This code was generated by a tool.
' Runtime Version:4.0.30319.42000
'
' Changes to this file may cause incorrect behavior and will be lost if
' the code is regenerated.
' </auto-generated>
'------------------------------------------------------------------------------
Option Strict On
Option Explicit On

View File

@@ -0,0 +1,10 @@
<?xml version="1.0" encoding="utf-8"?>
<MyApplicationData xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<MySubMain>false</MySubMain>
<SingleInstance>false</SingleInstance>
<ShutdownMode>0</ShutdownMode>
<EnableVisualStyles>true</EnableVisualStyles>
<AuthenticationMode>0</AuthenticationMode>
<ApplicationType>1</ApplicationType>
<SaveMySettingsOnExit>true</SaveMySettingsOnExit>
</MyApplicationData>

View File

@@ -0,0 +1,37 @@
Imports System.Resources
Imports System
Imports System.Reflection
Imports System.Runtime.InteropServices
' General Information about an assembly is controlled through the following
' set of attributes. Change these attribute values to modify the information
' associated with an assembly.
' Review the values of the assembly attributes
<Assembly: AssemblyTitle("LPSG")>
<Assembly: AssemblyDescription("LPSG plugin for SCrawler")>
<Assembly: AssemblyCompany("AndyProgram")>
<Assembly: AssemblyProduct("LPSG")>
<Assembly: AssemblyCopyright("Copyright © 2022")>
<Assembly: AssemblyTrademark("AndyProgram")>
<Assembly: ComVisible(False)>
'The following GUID is for the ID of the typelib if this project is exposed to COM
<Assembly: Guid("4beaf92d-0566-4331-ad29-034d09a7326f")>
' Version information for an assembly consists of the following four values:
'
' Major Version
' Minor Version
' Build Number
' Revision
'
' You can specify all the values or you can default the Build and Revision Numbers
' by using the '*' as shown below:
' <Assembly: AssemblyVersion("1.0.*")>
<Assembly: AssemblyVersion("1.0.0.2")>
<Assembly: AssemblyFileVersion("1.0.0.2")>
<Assembly: NeutralResourcesLanguage("en")>

View File

@@ -0,0 +1,83 @@
'------------------------------------------------------------------------------
' <auto-generated>
' This code was generated by a tool.
' Runtime Version:4.0.30319.42000
'
' Changes to this file may cause incorrect behavior and will be lost if
' the code is regenerated.
' </auto-generated>
'------------------------------------------------------------------------------
Option Strict On
Option Explicit On
Imports System
Namespace My.Resources
'This class was auto-generated by the StronglyTypedResourceBuilder
'class via a tool like ResGen or Visual Studio.
'To add or remove a member, edit your .ResX file then rerun ResGen
'with the /str option, or rebuild your VS project.
'''<summary>
''' A strongly-typed resource class, for looking up localized strings, etc.
'''</summary>
<Global.System.CodeDom.Compiler.GeneratedCodeAttribute("System.Resources.Tools.StronglyTypedResourceBuilder", "16.0.0.0"), _
Global.System.Diagnostics.DebuggerNonUserCodeAttribute(), _
Global.System.Runtime.CompilerServices.CompilerGeneratedAttribute(), _
Global.Microsoft.VisualBasic.HideModuleNameAttribute()> _
Friend Module Resources
Private resourceMan As Global.System.Resources.ResourceManager
Private resourceCulture As Global.System.Globalization.CultureInfo
'''<summary>
''' Returns the cached ResourceManager instance used by this class.
'''</summary>
<Global.System.ComponentModel.EditorBrowsableAttribute(Global.System.ComponentModel.EditorBrowsableState.Advanced)> _
Friend ReadOnly Property ResourceManager() As Global.System.Resources.ResourceManager
Get
If Object.ReferenceEquals(resourceMan, Nothing) Then
Dim temp As Global.System.Resources.ResourceManager = New Global.System.Resources.ResourceManager("LPSG.Resources", GetType(Resources).Assembly)
resourceMan = temp
End If
Return resourceMan
End Get
End Property
'''<summary>
''' Overrides the current thread's CurrentUICulture property for all
''' resource lookups using this strongly typed resource class.
'''</summary>
<Global.System.ComponentModel.EditorBrowsableAttribute(Global.System.ComponentModel.EditorBrowsableState.Advanced)> _
Friend Property Culture() As Global.System.Globalization.CultureInfo
Get
Return resourceCulture
End Get
Set
resourceCulture = value
End Set
End Property
'''<summary>
''' Looks up a localized resource of type System.Drawing.Icon similar to (Icon).
'''</summary>
Friend ReadOnly Property Icon32() As System.Drawing.Icon
Get
Dim obj As Object = ResourceManager.GetObject("Icon32", resourceCulture)
Return CType(obj,System.Drawing.Icon)
End Get
End Property
'''<summary>
''' Looks up a localized resource of type System.Drawing.Bitmap.
'''</summary>
Friend ReadOnly Property IconPic_32() As System.Drawing.Bitmap
Get
Dim obj As Object = ResourceManager.GetObject("IconPic_32", resourceCulture)
Return CType(obj,System.Drawing.Bitmap)
End Get
End Property
End Module
End Namespace

View File

@@ -0,0 +1,127 @@
<?xml version="1.0" encoding="utf-8"?>
<root>
<!--
Microsoft ResX Schema
Version 2.0
The primary goals of this format is to allow a simple XML format
that is mostly human readable. The generation and parsing of the
various data types are done through the TypeConverter classes
associated with the data types.
Example:
... ado.net/XML headers & schema ...
<resheader name="resmimetype">text/microsoft-resx</resheader>
<resheader name="version">2.0</resheader>
<resheader name="reader">System.Resources.ResXResourceReader, System.Windows.Forms, ...</resheader>
<resheader name="writer">System.Resources.ResXResourceWriter, System.Windows.Forms, ...</resheader>
<data name="Name1"><value>this is my long string</value><comment>this is a comment</comment></data>
<data name="Color1" type="System.Drawing.Color, System.Drawing">Blue</data>
<data name="Bitmap1" mimetype="application/x-microsoft.net.object.binary.base64">
<value>[base64 mime encoded serialized .NET Framework object]</value>
</data>
<data name="Icon1" type="System.Drawing.Icon, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
<value>[base64 mime encoded string representing a byte array form of the .NET Framework object]</value>
<comment>This is a comment</comment>
</data>
There are any number of "resheader" rows that contain simple
name/value pairs.
Each data row contains a name, and value. The row also contains a
type or mimetype. Type corresponds to a .NET class that support
text/value conversion through the TypeConverter architecture.
Classes that don't support this are serialized and stored with the
mimetype set.
The mimetype is used for serialized objects, and tells the
ResXResourceReader how to depersist the object. This is currently not
extensible. For a given mimetype the value must be set accordingly:
Note - application/x-microsoft.net.object.binary.base64 is the format
that the ResXResourceWriter will generate, however the reader can
read any of the formats listed below.
mimetype: application/x-microsoft.net.object.binary.base64
value : The object must be serialized with
: System.Runtime.Serialization.Formatters.Binary.BinaryFormatter
: and then encoded with base64 encoding.
mimetype: application/x-microsoft.net.object.soap.base64
value : The object must be serialized with
: System.Runtime.Serialization.Formatters.Soap.SoapFormatter
: and then encoded with base64 encoding.
mimetype: application/x-microsoft.net.object.bytearray.base64
value : The object must be serialized into a byte array
: using a System.ComponentModel.TypeConverter
: and then encoded with base64 encoding.
-->
<xsd:schema id="root" xmlns="" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:msdata="urn:schemas-microsoft-com:xml-msdata">
<xsd:import namespace="http://www.w3.org/XML/1998/namespace" />
<xsd:element name="root" msdata:IsDataSet="true">
<xsd:complexType>
<xsd:choice maxOccurs="unbounded">
<xsd:element name="metadata">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="value" type="xsd:string" minOccurs="0" />
</xsd:sequence>
<xsd:attribute name="name" use="required" type="xsd:string" />
<xsd:attribute name="type" type="xsd:string" />
<xsd:attribute name="mimetype" type="xsd:string" />
<xsd:attribute ref="xml:space" />
</xsd:complexType>
</xsd:element>
<xsd:element name="assembly">
<xsd:complexType>
<xsd:attribute name="alias" type="xsd:string" />
<xsd:attribute name="name" type="xsd:string" />
</xsd:complexType>
</xsd:element>
<xsd:element name="data">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="value" type="xsd:string" minOccurs="0" msdata:Ordinal="1" />
<xsd:element name="comment" type="xsd:string" minOccurs="0" msdata:Ordinal="2" />
</xsd:sequence>
<xsd:attribute name="name" type="xsd:string" use="required" msdata:Ordinal="1" />
<xsd:attribute name="type" type="xsd:string" msdata:Ordinal="3" />
<xsd:attribute name="mimetype" type="xsd:string" msdata:Ordinal="4" />
<xsd:attribute ref="xml:space" />
</xsd:complexType>
</xsd:element>
<xsd:element name="resheader">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="value" type="xsd:string" minOccurs="0" msdata:Ordinal="1" />
</xsd:sequence>
<xsd:attribute name="name" type="xsd:string" use="required" />
</xsd:complexType>
</xsd:element>
</xsd:choice>
</xsd:complexType>
</xsd:element>
</xsd:schema>
<resheader name="resmimetype">
<value>text/microsoft-resx</value>
</resheader>
<resheader name="version">
<value>2.0</value>
</resheader>
<resheader name="reader">
<value>System.Resources.ResXResourceReader, System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
</resheader>
<resheader name="writer">
<value>System.Resources.ResXResourceWriter, System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
</resheader>
<assembly alias="System.Windows.Forms" name="System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" />
<data name="Icon32" type="System.Resources.ResXFileRef, System.Windows.Forms">
<value>..\Content\Icons\Icon32.ico;System.Drawing.Icon, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a</value>
</data>
<data name="IconPic_32" type="System.Resources.ResXFileRef, System.Windows.Forms">
<value>..\Content\Pictures\IconPic_32.png;System.Drawing.Bitmap, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a</value>
</data>
</root>

View File

@@ -0,0 +1,73 @@
'------------------------------------------------------------------------------
' <auto-generated>
' This code was generated by a tool.
' Runtime Version:4.0.30319.42000
'
' Changes to this file may cause incorrect behavior and will be lost if
' the code is regenerated.
' </auto-generated>
'------------------------------------------------------------------------------
Option Strict On
Option Explicit On
Namespace My
<Global.System.Runtime.CompilerServices.CompilerGeneratedAttribute(), _
Global.System.CodeDom.Compiler.GeneratedCodeAttribute("Microsoft.VisualStudio.Editors.SettingsDesigner.SettingsSingleFileGenerator", "16.10.0.0"), _
Global.System.ComponentModel.EditorBrowsableAttribute(Global.System.ComponentModel.EditorBrowsableState.Advanced)> _
Partial Friend NotInheritable Class MySettings
Inherits Global.System.Configuration.ApplicationSettingsBase
Private Shared defaultInstance As MySettings = CType(Global.System.Configuration.ApplicationSettingsBase.Synchronized(New MySettings()),MySettings)
#Region "My.Settings Auto-Save Functionality"
#If _MyType = "WindowsForms" Then
Private Shared addedHandler As Boolean
Private Shared addedHandlerLockObject As New Object
<Global.System.Diagnostics.DebuggerNonUserCodeAttribute(), Global.System.ComponentModel.EditorBrowsableAttribute(Global.System.ComponentModel.EditorBrowsableState.Advanced)> _
Private Shared Sub AutoSaveSettings(sender As Global.System.Object, e As Global.System.EventArgs)
If My.Application.SaveMySettingsOnExit Then
My.Settings.Save()
End If
End Sub
#End If
#End Region
Public Shared ReadOnly Property [Default]() As MySettings
Get
#If _MyType = "WindowsForms" Then
If Not addedHandler Then
SyncLock addedHandlerLockObject
If Not addedHandler Then
AddHandler My.Application.Shutdown, AddressOf AutoSaveSettings
addedHandler = True
End If
End SyncLock
End If
#End If
Return defaultInstance
End Get
End Property
End Class
End Namespace
Namespace My
<Global.Microsoft.VisualBasic.HideModuleNameAttribute(), _
Global.System.Diagnostics.DebuggerNonUserCodeAttribute(), _
Global.System.Runtime.CompilerServices.CompilerGeneratedAttribute()> _
Friend Module MySettingsProperty
<Global.System.ComponentModel.Design.HelpKeywordAttribute("My.Settings")> _
Friend ReadOnly Property Settings() As Global.LPSG.My.MySettings
Get
Return Global.LPSG.My.MySettings.Default
End Get
End Property
End Module
End Namespace

View File

@@ -0,0 +1,7 @@
<?xml version='1.0' encoding='utf-8'?>
<SettingsFile xmlns="http://schemas.microsoft.com/VisualStudio/2004/01/settings" CurrentProfile="(Default)" UseMySettingsClassName="true">
<Profiles>
<Profile Name="(Default)" />
</Profiles>
<Settings />
</SettingsFile>

View File

@@ -0,0 +1,166 @@
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="15.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Import Project="$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props" Condition="Exists('$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props')" />
<PropertyGroup>
<Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
<Platform Condition=" '$(Platform)' == '' ">AnyCPU</Platform>
<ProjectGuid>{22A130B2-DDF4-4FB5-BA38-E5DB4CF1B8A2}</ProjectGuid>
<OutputType>Library</OutputType>
<RootNamespace>LPSG</RootNamespace>
<AssemblyName>SCrawler.Plugin.LPSG</AssemblyName>
<FileAlignment>512</FileAlignment>
<MyType>Windows</MyType>
<TargetFrameworkVersion>v4.6.1</TargetFrameworkVersion>
<Deterministic>true</Deterministic>
</PropertyGroup>
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
<DebugSymbols>true</DebugSymbols>
<DebugType>full</DebugType>
<DefineDebug>true</DefineDebug>
<DefineTrace>true</DefineTrace>
<OutputPath>bin\Debug\</OutputPath>
<DocumentationFile>
</DocumentationFile>
<NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
</PropertyGroup>
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
<DebugType>pdbonly</DebugType>
<DefineDebug>false</DefineDebug>
<DefineTrace>true</DefineTrace>
<Optimize>true</Optimize>
<OutputPath>bin\Release\</OutputPath>
<DocumentationFile>
</DocumentationFile>
<NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
</PropertyGroup>
<PropertyGroup>
<OptionExplicit>On</OptionExplicit>
</PropertyGroup>
<PropertyGroup>
<OptionCompare>Binary</OptionCompare>
</PropertyGroup>
<PropertyGroup>
<OptionStrict>Off</OptionStrict>
</PropertyGroup>
<PropertyGroup>
<OptionInfer>On</OptionInfer>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Debug|x64'">
<DebugSymbols>true</DebugSymbols>
<DefineDebug>true</DefineDebug>
<DefineTrace>true</DefineTrace>
<OutputPath>bin\x64\Debug\</OutputPath>
<NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
<DebugType>full</DebugType>
<PlatformTarget>x64</PlatformTarget>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Release|x64'">
<DefineTrace>true</DefineTrace>
<OutputPath>bin\x64\Release\</OutputPath>
<Optimize>true</Optimize>
<NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
<DebugType>pdbonly</DebugType>
<PlatformTarget>x64</PlatformTarget>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Debug|x86'">
<DebugSymbols>true</DebugSymbols>
<DefineDebug>true</DefineDebug>
<DefineTrace>true</DefineTrace>
<OutputPath>bin\x86\Debug\</OutputPath>
<NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
<DebugType>full</DebugType>
<PlatformTarget>x86</PlatformTarget>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Release|x86'">
<DefineTrace>true</DefineTrace>
<OutputPath>bin\x86\Release\</OutputPath>
<Optimize>true</Optimize>
<NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
<DebugType>pdbonly</DebugType>
<PlatformTarget>x86</PlatformTarget>
</PropertyGroup>
<ItemGroup>
<Reference Include="System" />
<Reference Include="System.Data" />
<Reference Include="System.Drawing" />
<Reference Include="System.Windows.Forms" />
<Reference Include="System.Xml" />
<Reference Include="System.Core" />
<Reference Include="System.Xml.Linq" />
<Reference Include="System.Data.DataSetExtensions" />
<Reference Include="System.Net.Http" />
</ItemGroup>
<ItemGroup>
<Import Include="Microsoft.VisualBasic" />
<Import Include="PersonalUtilities.Functions" />
<Import Include="PersonalUtilities.Tools.WEB" />
<Import Include="SCrawler.Plugin" />
<Import Include="SCrawler.Plugin.Attributes" />
<Import Include="System" />
<Import Include="System.Collections" />
<Import Include="System.Collections.Generic" />
<Import Include="System.Data" />
<Import Include="System.Diagnostics" />
<Import Include="System.Linq" />
<Import Include="System.Xml.Linq" />
<Import Include="System.Threading.Tasks" />
</ItemGroup>
<ItemGroup>
<Compile Include="Declarations.vb" />
<Compile Include="My Project\AssemblyInfo.vb" />
<Compile Include="My Project\Application.Designer.vb">
<AutoGen>True</AutoGen>
<DependentUpon>Application.myapp</DependentUpon>
<DesignTime>True</DesignTime>
</Compile>
<Compile Include="My Project\Resources.Designer.vb">
<AutoGen>True</AutoGen>
<DesignTime>True</DesignTime>
<DependentUpon>Resources.resx</DependentUpon>
</Compile>
<Compile Include="My Project\Settings.Designer.vb">
<AutoGen>True</AutoGen>
<DependentUpon>Settings.settings</DependentUpon>
<DesignTimeSharedInput>True</DesignTimeSharedInput>
</Compile>
<Compile Include="SiteSettings.vb" />
<Compile Include="UserData.vb" />
</ItemGroup>
<ItemGroup>
<EmbeddedResource Include="My Project\Resources.resx">
<Generator>VbMyResourcesResXFileCodeGenerator</Generator>
<LastGenOutput>Resources.Designer.vb</LastGenOutput>
<CustomToolNamespace>My.Resources</CustomToolNamespace>
<SubType>Designer</SubType>
</EmbeddedResource>
</ItemGroup>
<ItemGroup>
<None Include=".editorconfig" />
<None Include="My Project\Application.myapp">
<Generator>MyApplicationCodeGenerator</Generator>
<LastGenOutput>Application.Designer.vb</LastGenOutput>
</None>
<None Include="My Project\Settings.settings">
<Generator>SettingsSingleFileGenerator</Generator>
<CustomToolNamespace>My</CustomToolNamespace>
<LastGenOutput>Settings.Designer.vb</LastGenOutput>
</None>
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\..\..\MyUtilities\PersonalUtilities\PersonalUtilities.vbproj">
<Project>{8405896b-2685-4916-bc93-1fb514c323a9}</Project>
<Name>PersonalUtilities</Name>
</ProjectReference>
<ProjectReference Include="..\SCrawler.PluginProvider\SCrawler.PluginProvider.vbproj">
<Project>{d4650f6b-5a54-44b6-999b-6c675b7116b1}</Project>
<Name>SCrawler.PluginProvider</Name>
</ProjectReference>
</ItemGroup>
<ItemGroup>
<None Include="Content\Icons\Icon32.ico" />
</ItemGroup>
<ItemGroup>
<None Include="Content\Pictures\IconPic_32.png" />
</ItemGroup>
<Import Project="$(MSBuildToolsPath)\Microsoft.VisualBasic.targets" />
</Project>

View File

@@ -0,0 +1,100 @@
' Copyright (C) 2022 Andy
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
' (at your option) any later version.
'
' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY
Imports System.Drawing
Imports PersonalUtilities.Functions.RegularExpressions
<Manifest("AndyProgram_LPSG")>
Public Class SiteSettings : Implements ISiteSettings
Public ReadOnly Property Icon As Icon Implements ISiteSettings.Icon
Get
Return My.Resources.Icon32
End Get
End Property
Public ReadOnly Property Image As Image Implements ISiteSettings.Image
Get
Return My.Resources.IconPic_32
End Get
End Property
Public ReadOnly Property Site As String = "LPSG" Implements ISiteSettings.Site
Public Property Logger As ILogProvider Implements ISiteSettings.Logger
Friend ReadOnly Property Responser As Response
Public Sub New()
Responser = New Response($"Settings\Responser_LPSG.xml")
With Responser
If .File.Exists Then
.LoadSettings()
Else
.CookiesDomain = "www.lpsg.com"
.Cookies = New CookieKeeper("www.lpsg.com")
End If
End With
End Sub
Public Sub BeginInit() Implements ISiteSettings.BeginInit
End Sub
Public Sub EndInit() Implements ISiteSettings.EndInit
End Sub
Public Function GetInstance(ByVal What As ISiteSettings.Download) As IPluginContentProvider Implements ISiteSettings.GetInstance
Return New UserData
End Function
Public Sub Load(ByVal XMLValues As IEnumerable(Of KeyValuePair(Of String, String))) Implements ISiteSettings.Load
End Sub
#Region "Download functions"
Public Sub DownloadStarted(ByVal What As ISiteSettings.Download) Implements ISiteSettings.DownloadStarted
End Sub
Public Sub BeforeStartDownload(ByVal User As Object, ByVal What As ISiteSettings.Download) Implements ISiteSettings.BeforeStartDownload
End Sub
Public Sub AfterDownload(ByVal User As Object, ByVal What As ISiteSettings.Download) Implements ISiteSettings.AfterDownload
End Sub
Public Sub DownloadDone(ByVal What As ISiteSettings.Download) Implements ISiteSettings.DownloadDone
End Sub
#End Region
#Region "Update"
Public Sub BeginUpdate() Implements ISiteSettings.BeginUpdate
End Sub
Public Sub EndUpdate() Implements ISiteSettings.EndUpdate
End Sub
Public Sub Update() Implements ISiteSettings.Update
Responser.SaveSettings()
End Sub
#End Region
Public Sub Reset() Implements ISiteSettings.Reset
End Sub
Public Sub OpenSettingsForm() Implements ISiteSettings.OpenSettingsForm
End Sub
Public Sub UserOptions(ByRef Options As Object, ByVal OpenForm As Boolean) Implements ISiteSettings.UserOptions
Options = Nothing
End Sub
Public Function GetUserUrl(ByVal UserName As String, ByVal Channel As Boolean) As String Implements ISiteSettings.GetUserUrl
Return $"https://www.lpsg.com/threads/{UserName}/"
End Function
Private ReadOnly UserRegEx As RParams = RParams.DMS(".+?lpsg.com/threads/([^/]+)", 1)
Public Function IsMyUser(ByVal UserURL As String) As ExchangeOptions Implements ISiteSettings.IsMyUser
Try
Dim r$ = RegexReplace(UserURL, UserRegEx)
If Not r.IsEmptyString Then
Return New ExchangeOptions(Site, r)
Else
Return Nothing
End If
Catch ex As Exception
Return Nothing
End Try
End Function
Public Function IsMyImageVideo(ByVal URL As String) As ExchangeOptions Implements ISiteSettings.IsMyImageVideo
Return Nothing
End Function
Public Function GetSpecialData(ByVal URL As String, ByVal Path As String, ByVal AskForPath As Boolean) As IEnumerable(Of PluginUserMedia) Implements ISiteSettings.GetSpecialData
Return Nothing
End Function
Public Function Available(ByVal What As ISiteSettings.Download) As Boolean Implements ISiteSettings.Available
Return True
End Function
Public Function ReadyToDownload(ByVal What As ISiteSettings.Download) As Boolean Implements ISiteSettings.ReadyToDownload
Return True
End Function
End Class

View File

@@ -0,0 +1,190 @@
' Copyright (C) 2022 Andy
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
' (at your option) any later version.
'
' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY
Imports PersonalUtilities.Functions.RegularExpressions
Imports UStates = SCrawler.Plugin.PluginUserMedia.States
Imports UTypes = SCrawler.Plugin.PluginUserMedia.Types
Imports Converters = PersonalUtilities.Functions.SymbolsConverter.Converters
Public Class UserData : Implements IPluginContentProvider
#Region "XML names"
Private Const Name_LatestPage As String = "LatestPage"
#End Region
#Region "Interface declarations"
Public Event ProgressChanged(ByVal Count As Integer) Implements IPluginContentProvider.ProgressChanged
Public Event TotalCountChanged(ByVal Count As Integer) Implements IPluginContentProvider.TotalCountChanged
Public Property Thrower As IThrower Implements IPluginContentProvider.Thrower
Public Property LogProvider As ILogProvider Implements IPluginContentProvider.LogProvider
Public Property ESettings As ISiteSettings Implements IPluginContentProvider.Settings
Private ReadOnly Property Settings As SiteSettings
Get
Return DirectCast(ESettings, SiteSettings)
End Get
End Property
Public Property Name As String Implements IPluginContentProvider.Name
Public Property ID As String Implements IPluginContentProvider.ID
Public Property ParseUserMediaOnly As Boolean Implements IPluginContentProvider.ParseUserMediaOnly
Public Property UserDescription As String Implements IPluginContentProvider.UserDescription
Public Property ExistingContentList As List(Of PluginUserMedia) Implements IPluginContentProvider.ExistingContentList
Public Property TempPostsList As List(Of String) Implements IPluginContentProvider.TempPostsList
Public Property TempMediaList As List(Of PluginUserMedia) Implements IPluginContentProvider.TempMediaList
Public Property UserExists As Boolean Implements IPluginContentProvider.UserExists
Public Property UserSuspended As Boolean Implements IPluginContentProvider.UserSuspended
Public Property IsSavedPosts As Boolean Implements IPluginContentProvider.IsSavedPosts
Public Property SeparateVideoFolder As Boolean Implements IPluginContentProvider.SeparateVideoFolder
Public Property DataPath As String Implements IPluginContentProvider.DataPath
Public Property PostsNumberLimit As Integer? Implements IPluginContentProvider.PostsNumberLimit
Public Property PostsDateLimit As Date? Implements IPluginContentProvider.PostsDateLimit
#End Region
#Region "Interface exchange options"
Public Sub ExchangeOptionsSet(ByVal Obj As Object) Implements IPluginContentProvider.ExchangeOptionsSet
End Sub
Public Function ExchangeOptionsGet() As Object Implements IPluginContentProvider.ExchangeOptionsGet
Return Nothing
End Function
#End Region
#Region "Interface XML"
Public Sub XmlFieldsSet(ByVal Fields As List(Of KeyValuePair(Of String, String))) Implements IPluginContentProvider.XmlFieldsSet
If Fields.ListExists Then
For Each f As KeyValuePair(Of String, String) In Fields
If f.Key = Name_LatestPage Then LatestPage = f.Value
Next
End If
End Sub
Public Function XmlFieldsGet() As List(Of KeyValuePair(Of String, String)) Implements IPluginContentProvider.XmlFieldsGet
Return New List(Of KeyValuePair(Of String, String)) From {New KeyValuePair(Of String, String)(Name_LatestPage, LatestPage)}
End Function
#End Region
Private Property LatestPage As String = String.Empty
Private Property Responser As Response = Nothing
Private Enum Mode : Internal : External : End Enum
Public Sub GetMedia() Implements IPluginContentProvider.GetMedia
Try
If Not Responser Is Nothing Then Responser.Dispose()
Responser = New Response
With Responser : .Copy(Settings.Responser) : .Error = EDP.ThrowException : End With
Dim NextPage$
Dim r$
Dim _LPage As Func(Of String) = Function() If(LatestPage.IsEmptyString, String.Empty, $"page-{LatestPage}")
Do
r = Responser.GetResponse($"https://www.lpsg.com/threads/{Name}/{_LPage.Invoke}")
UserExists = True
UserSuspended = False
Thrower.ThrowAny()
If Not r.IsEmptyString Then
NextPage = RegexReplace(r, NextPageRegex)
UpdateMediaList(RegexReplace(r, PhotoRegEx), Mode.Internal)
UpdateMediaList(RegexReplace(r, PhotoRegExExt), Mode.External)
If NextPage = LatestPage Or NextPage.IsEmptyString Then Exit Do Else LatestPage = NextPage
Else
Exit Do
End If
Loop
If TempMediaList.ListExists And ExistingContentList.ListExists Then _
TempMediaList.RemoveAll(Function(m) ExistingContentList.Exists(Function(mm) mm.URL = m.URL))
Catch oex As OperationCanceledException
Catch dex As ObjectDisposedException
Catch ex As Exception
If Responser.StatusCode = Net.HttpStatusCode.ServiceUnavailable Then
LogProvider.Add("LPSG not available")
Else
LogProvider.Add(ex, "[LPSG.UserData.GetMedia]")
End If
End Try
End Sub
Private Sub UpdateMediaList(ByVal l As List(Of String), ByVal m As Mode)
If l.ListExists Then
Dim f As SFile
Dim u$
Dim exists As Boolean
Dim r As RParams
Dim ude As New ErrorsDescriber(EDP.ReturnValue)
For Each url$ In l
If Not url.IsEmptyString Then u = SymbolsConverter.Decode(url, {Converters.HTML, Converters.ASCII}, ude) Else u = String.Empty
If Not u.IsEmptyString Then
exists = Not IsEmptyString(RegexReplace(u, FileExistsRegEx))
If m = Mode.Internal Then
r = FileRegEx
Else
r = FileRegExExt
If Not exists Then
r = FileRegExExt2
exists = Not IsEmptyString(RegexReplace(u, FileRegExExt2))
End If
End If
If exists Then
f = CStr(RegexReplace(u, r))
f.Path = DataPath.CSFilePSN
f.Separator = "\"
If f.Extension.IsEmptyString Then f.Extension = "jpg"
TempMediaList.ListAddValue(New PluginUserMedia With {.ContentType = UTypes.Picture, .URL = url, .File = f}, TempListAddParams)
End If
End If
Next
End If
End Sub
Public Sub Download() Implements IPluginContentProvider.Download
Try
With Responser : .UseWebClient = True : .UseWebClientCookies = True : .ResetError() : End With
If TempMediaList.ListExists Then
Dim m As PluginUserMedia
Dim eweb As ErrorsDescriber = EDP.ThrowException
RaiseEvent TotalCountChanged(TempMediaList.Count)
For i% = 0 To TempMediaList.Count - 1
Thrower.ThrowAny()
m = TempMediaList(i)
m.DownloadState = UStates.Tried
Try
If Not m.URL.IsEmptyString And Not m.File.IsEmptyString Then
Responser.DownloadFile(m.URL, m.File, eweb)
m.DownloadState = UStates.Downloaded
Else
m.DownloadState = UStates.Skipped
End If
Catch wex As Exception
If Responser.Client.StatusCode = Net.HttpStatusCode.ServiceUnavailable Then
LogProvider.Add("LPSG not available")
Else
m.DownloadState = UStates.Skipped
End If
End Try
RaiseEvent ProgressChanged(1)
TempMediaList(i) = m
Next
End If
Catch oex As OperationCanceledException
Catch dex As ObjectDisposedException
Catch ex As Exception
LogProvider.Add(ex, "[LPSG.UserData.Download]")
End Try
End Sub
#Region "IDisposable Support"
Private disposedValue As Boolean = False
Protected Overridable Overloads Sub Dispose(ByVal disposing As Boolean)
If Not disposedValue Then
If disposing Then
If Not Responser Is Nothing Then Responser.Dispose()
If ExistingContentList.ListExists Then ExistingContentList.Clear()
If TempPostsList.ListExists Then TempPostsList.Clear()
If TempMediaList.ListExists Then TempMediaList.Clear()
End If
disposedValue = True
End If
End Sub
Protected Overrides Sub Finalize()
Dispose(False)
MyBase.Finalize()
End Sub
Public Overloads Sub Dispose() Implements IDisposable.Dispose
Dispose(True)
GC.SuppressFinalize(Me)
End Sub
#End Region
End Class

View File

@@ -0,0 +1,3 @@
[*.vb]
# Modifier preferences
file_header_template = Copyright (C) 2022 Andy\nThis program is free software: you can redistribute it and/or modify\nit under the terms of the GNU General Public License as published by\nthe Free Software Foundation, either version 3 of the License, or\n(at your option) any later version.\n\nThis program is distributed in the hope that it will be useful,\nbut WITHOUT ANY WARRANTY; without even the implied warranty of\nMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the\nGNU General Public License for more details.\n\nYou should have received a copy of the GNU General Public License\nalong with this program. If not, see <https://www.gnu.org/licenses/>

Binary file not shown.

After

Width:  |  Height:  |  Size: 17 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1016 B

View File

@@ -0,0 +1,16 @@
' Copyright (C) 2022 Andy
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
' (at your option) any later version.
'
' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY
Imports PersonalUtilities.Functions.RegularExpressions
Friend Module Declarations
Friend ReadOnly Property M3U8Regex As RParams = RParams.DM("http.+?.m3u8.*?(?=')", 0)
Friend ReadOnly Property VideoTitleRegex As RParams = RParams.DMS("html5player.setVideoTitle\('(.+)(?='\);)", 1)
Friend ReadOnly Property VideoID As RParams = RParams.DMS(".*?www.xvideos.com/(video\d+).*", 1)
Friend ReadOnly Property M3U8Reparse As RParams = RParams.DM("NAME=""(\d+).*?""[\r\n]*?(.+)(?=(|[\r\n]+?))", 0, RegexReturn.List)
Friend ReadOnly Property M3U8Appender As RParams = RParams.DM("(.+)(?=/.+?\.m3u8.*?)", 0)
End Module

View File

@@ -0,0 +1,69 @@
' Copyright (C) 2022 Andy
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
' (at your option) any later version.
'
' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY
Imports System.Net
Imports PersonalUtilities.Tools.WEB
Friend NotInheritable Class M3U8
Private Sub New()
End Sub
Private Shared Function Save(ByVal URLs As List(Of String), ByVal ffmpegFile As SFile, ByVal f As SFile, ByRef Logger As ILogProvider) As SFile
Dim CachePath As SFile = Nothing
Try
If URLs.ListExists Then
Dim ConcatFile As SFile = f
If ConcatFile.Name.IsEmptyString Then ConcatFile.Name = "PlayListFile"
ConcatFile.Extension = "mp4"
CachePath = $"{f.PathWithSeparator}_Cache\{SFile.GetDirectories($"{f.PathWithSeparator}_Cache\",,, EDP.ReturnValue).ListIfNothing.Count + 1}\"
If CachePath.Exists(SFO.Path) Then
Dim p As New SFileNumbers(ConcatFile.Name,,, New ANumbers With {.Format = ANumbers.Formats.General})
ConcatFile = SFile.Indexed_IndexFile(ConcatFile,, p, EDP.ReturnValue)
Dim i%
Dim eFiles As New List(Of SFile)
Dim dFile As SFile = CachePath
dFile.Extension = "ts"
Using w As New WebClient
For i = 0 To URLs.Count - 1
dFile.Name = $"ConPart_{i}"
w.DownloadFile(URLs(i), dFile)
eFiles.Add(dFile)
Next
End Using
f = FFMPEG.ConcatenateFiles(eFiles, ffmpegFile, ConcatFile, p, EDP.ThrowException)
eFiles.Clear()
Return f
End If
End If
Return Nothing
Catch ex As Exception
Logger.Add(ex, "[M3U8.Save]")
ex.HelpLink = 1
Throw ex
Finally
CachePath.Delete(SFO.Path, SFODelete.None, EDP.None)
End Try
End Function
Friend Shared Function Download(ByVal URL As String, ByVal Appender As String, ByVal ffmpegFile As SFile, ByVal f As SFile,
ByRef Logger As ILogProvider) As SFile
Try
If Not URL.IsEmptyString Then
Using w As New WebClient
Dim r$ = w.DownloadString(URL)
If Not r.IsEmptyString Then
Dim l As List(Of String) = ListAddList(Nothing, r.StringFormatLines.StringToList(Of String)(vbNewLine).ListWithRemove(Function(v) v.Trim.StartsWith("#")),
New ListAddParams With {.Converter = Function(Input) $"{Appender}/{Input.ToString.Trim}"})
If l.ListExists Then Return Save(l, ffmpegFile, f, Logger)
End If
End Using
End If
Return Nothing
Catch ex As Exception
If Not ex.HelpLink = 1 Then Logger.Add(ex, "[M3U8.Download]")
Throw ex
End Try
End Function
End Class

View File

@@ -0,0 +1,13 @@
'------------------------------------------------------------------------------
' <auto-generated>
' This code was generated by a tool.
' Runtime Version:4.0.30319.42000
'
' Changes to this file may cause incorrect behavior and will be lost if
' the code is regenerated.
' </auto-generated>
'------------------------------------------------------------------------------
Option Strict On
Option Explicit On

View File

@@ -0,0 +1,10 @@
<?xml version="1.0" encoding="utf-8"?>
<MyApplicationData xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<MySubMain>false</MySubMain>
<SingleInstance>false</SingleInstance>
<ShutdownMode>0</ShutdownMode>
<EnableVisualStyles>true</EnableVisualStyles>
<AuthenticationMode>0</AuthenticationMode>
<ApplicationType>1</ApplicationType>
<SaveMySettingsOnExit>true</SaveMySettingsOnExit>
</MyApplicationData>

View File

@@ -0,0 +1,37 @@
Imports System.Resources
Imports System
Imports System.Reflection
Imports System.Runtime.InteropServices
' General Information about an assembly is controlled through the following
' set of attributes. Change these attribute values to modify the information
' associated with an assembly.
' Review the values of the assembly attributes
<Assembly: AssemblyTitle("XVIDEOS")>
<Assembly: AssemblyDescription("XVIDEOS plugin for SCrawler")>
<Assembly: AssemblyCompany("AndyProgram")>
<Assembly: AssemblyProduct("XVIDEOS")>
<Assembly: AssemblyCopyright("Copyright © 2022")>
<Assembly: AssemblyTrademark("AndyProgram")>
<Assembly: ComVisible(False)>
'The following GUID is for the ID of the typelib if this project is exposed to COM
<Assembly: Guid("d31a0e3b-081d-4155-9a41-4c0a641c87b6")>
' Version information for an assembly consists of the following four values:
'
' Major Version
' Minor Version
' Build Number
' Revision
'
' You can specify all the values or you can default the Build and Revision Numbers
' by using the '*' as shown below:
' <Assembly: AssemblyVersion("1.0.*")>
<Assembly: AssemblyVersion("1.0.0.2")>
<Assembly: AssemblyFileVersion("1.0.0.2")>
<Assembly: NeutralResourcesLanguage("en")>

View File

@@ -0,0 +1,83 @@
'------------------------------------------------------------------------------
' <auto-generated>
' This code was generated by a tool.
' Runtime Version:4.0.30319.42000
'
' Changes to this file may cause incorrect behavior and will be lost if
' the code is regenerated.
' </auto-generated>
'------------------------------------------------------------------------------
Option Strict On
Option Explicit On
Imports System
Namespace My.Resources
'This class was auto-generated by the StronglyTypedResourceBuilder
'class via a tool like ResGen or Visual Studio.
'To add or remove a member, edit your .ResX file then rerun ResGen
'with the /str option, or rebuild your VS project.
'''<summary>
''' A strongly-typed resource class, for looking up localized strings, etc.
'''</summary>
<Global.System.CodeDom.Compiler.GeneratedCodeAttribute("System.Resources.Tools.StronglyTypedResourceBuilder", "16.0.0.0"), _
Global.System.Diagnostics.DebuggerNonUserCodeAttribute(), _
Global.System.Runtime.CompilerServices.CompilerGeneratedAttribute(), _
Global.Microsoft.VisualBasic.HideModuleNameAttribute()> _
Friend Module Resources
Private resourceMan As Global.System.Resources.ResourceManager
Private resourceCulture As Global.System.Globalization.CultureInfo
'''<summary>
''' Returns the cached ResourceManager instance used by this class.
'''</summary>
<Global.System.ComponentModel.EditorBrowsableAttribute(Global.System.ComponentModel.EditorBrowsableState.Advanced)> _
Friend ReadOnly Property ResourceManager() As Global.System.Resources.ResourceManager
Get
If Object.ReferenceEquals(resourceMan, Nothing) Then
Dim temp As Global.System.Resources.ResourceManager = New Global.System.Resources.ResourceManager("XVIDEOS.Resources", GetType(Resources).Assembly)
resourceMan = temp
End If
Return resourceMan
End Get
End Property
'''<summary>
''' Overrides the current thread's CurrentUICulture property for all
''' resource lookups using this strongly typed resource class.
'''</summary>
<Global.System.ComponentModel.EditorBrowsableAttribute(Global.System.ComponentModel.EditorBrowsableState.Advanced)> _
Friend Property Culture() As Global.System.Globalization.CultureInfo
Get
Return resourceCulture
End Get
Set
resourceCulture = value
End Set
End Property
'''<summary>
''' Looks up a localized resource of type System.Drawing.Icon similar to (Icon).
'''</summary>
Friend ReadOnly Property Icon32() As System.Drawing.Icon
Get
Dim obj As Object = ResourceManager.GetObject("Icon32", resourceCulture)
Return CType(obj,System.Drawing.Icon)
End Get
End Property
'''<summary>
''' Looks up a localized resource of type System.Drawing.Bitmap.
'''</summary>
Friend ReadOnly Property IconPic32() As System.Drawing.Bitmap
Get
Dim obj As Object = ResourceManager.GetObject("IconPic32", resourceCulture)
Return CType(obj,System.Drawing.Bitmap)
End Get
End Property
End Module
End Namespace

View File

@@ -0,0 +1,127 @@
<?xml version="1.0" encoding="utf-8"?>
<root>
<!--
Microsoft ResX Schema
Version 2.0
The primary goals of this format is to allow a simple XML format
that is mostly human readable. The generation and parsing of the
various data types are done through the TypeConverter classes
associated with the data types.
Example:
... ado.net/XML headers & schema ...
<resheader name="resmimetype">text/microsoft-resx</resheader>
<resheader name="version">2.0</resheader>
<resheader name="reader">System.Resources.ResXResourceReader, System.Windows.Forms, ...</resheader>
<resheader name="writer">System.Resources.ResXResourceWriter, System.Windows.Forms, ...</resheader>
<data name="Name1"><value>this is my long string</value><comment>this is a comment</comment></data>
<data name="Color1" type="System.Drawing.Color, System.Drawing">Blue</data>
<data name="Bitmap1" mimetype="application/x-microsoft.net.object.binary.base64">
<value>[base64 mime encoded serialized .NET Framework object]</value>
</data>
<data name="Icon1" type="System.Drawing.Icon, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
<value>[base64 mime encoded string representing a byte array form of the .NET Framework object]</value>
<comment>This is a comment</comment>
</data>
There are any number of "resheader" rows that contain simple
name/value pairs.
Each data row contains a name, and value. The row also contains a
type or mimetype. Type corresponds to a .NET class that support
text/value conversion through the TypeConverter architecture.
Classes that don't support this are serialized and stored with the
mimetype set.
The mimetype is used for serialized objects, and tells the
ResXResourceReader how to depersist the object. This is currently not
extensible. For a given mimetype the value must be set accordingly:
Note - application/x-microsoft.net.object.binary.base64 is the format
that the ResXResourceWriter will generate, however the reader can
read any of the formats listed below.
mimetype: application/x-microsoft.net.object.binary.base64
value : The object must be serialized with
: System.Runtime.Serialization.Formatters.Binary.BinaryFormatter
: and then encoded with base64 encoding.
mimetype: application/x-microsoft.net.object.soap.base64
value : The object must be serialized with
: System.Runtime.Serialization.Formatters.Soap.SoapFormatter
: and then encoded with base64 encoding.
mimetype: application/x-microsoft.net.object.bytearray.base64
value : The object must be serialized into a byte array
: using a System.ComponentModel.TypeConverter
: and then encoded with base64 encoding.
-->
<xsd:schema id="root" xmlns="" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:msdata="urn:schemas-microsoft-com:xml-msdata">
<xsd:import namespace="http://www.w3.org/XML/1998/namespace" />
<xsd:element name="root" msdata:IsDataSet="true">
<xsd:complexType>
<xsd:choice maxOccurs="unbounded">
<xsd:element name="metadata">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="value" type="xsd:string" minOccurs="0" />
</xsd:sequence>
<xsd:attribute name="name" use="required" type="xsd:string" />
<xsd:attribute name="type" type="xsd:string" />
<xsd:attribute name="mimetype" type="xsd:string" />
<xsd:attribute ref="xml:space" />
</xsd:complexType>
</xsd:element>
<xsd:element name="assembly">
<xsd:complexType>
<xsd:attribute name="alias" type="xsd:string" />
<xsd:attribute name="name" type="xsd:string" />
</xsd:complexType>
</xsd:element>
<xsd:element name="data">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="value" type="xsd:string" minOccurs="0" msdata:Ordinal="1" />
<xsd:element name="comment" type="xsd:string" minOccurs="0" msdata:Ordinal="2" />
</xsd:sequence>
<xsd:attribute name="name" type="xsd:string" use="required" msdata:Ordinal="1" />
<xsd:attribute name="type" type="xsd:string" msdata:Ordinal="3" />
<xsd:attribute name="mimetype" type="xsd:string" msdata:Ordinal="4" />
<xsd:attribute ref="xml:space" />
</xsd:complexType>
</xsd:element>
<xsd:element name="resheader">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="value" type="xsd:string" minOccurs="0" msdata:Ordinal="1" />
</xsd:sequence>
<xsd:attribute name="name" type="xsd:string" use="required" />
</xsd:complexType>
</xsd:element>
</xsd:choice>
</xsd:complexType>
</xsd:element>
</xsd:schema>
<resheader name="resmimetype">
<value>text/microsoft-resx</value>
</resheader>
<resheader name="version">
<value>2.0</value>
</resheader>
<resheader name="reader">
<value>System.Resources.ResXResourceReader, System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
</resheader>
<resheader name="writer">
<value>System.Resources.ResXResourceWriter, System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
</resheader>
<assembly alias="System.Windows.Forms" name="System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" />
<data name="Icon32" type="System.Resources.ResXFileRef, System.Windows.Forms">
<value>..\Content\Icons\Icon32.ico;System.Drawing.Icon, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a</value>
</data>
<data name="IconPic32" type="System.Resources.ResXFileRef, System.Windows.Forms">
<value>..\Content\Pictures\IconPic32.png;System.Drawing.Bitmap, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a</value>
</data>
</root>

View File

@@ -0,0 +1,73 @@
'------------------------------------------------------------------------------
' <auto-generated>
' This code was generated by a tool.
' Runtime Version:4.0.30319.42000
'
' Changes to this file may cause incorrect behavior and will be lost if
' the code is regenerated.
' </auto-generated>
'------------------------------------------------------------------------------
Option Strict On
Option Explicit On
Namespace My
<Global.System.Runtime.CompilerServices.CompilerGeneratedAttribute(), _
Global.System.CodeDom.Compiler.GeneratedCodeAttribute("Microsoft.VisualStudio.Editors.SettingsDesigner.SettingsSingleFileGenerator", "16.10.0.0"), _
Global.System.ComponentModel.EditorBrowsableAttribute(Global.System.ComponentModel.EditorBrowsableState.Advanced)> _
Partial Friend NotInheritable Class MySettings
Inherits Global.System.Configuration.ApplicationSettingsBase
Private Shared defaultInstance As MySettings = CType(Global.System.Configuration.ApplicationSettingsBase.Synchronized(New MySettings()),MySettings)
#Region "My.Settings Auto-Save Functionality"
#If _MyType = "WindowsForms" Then
Private Shared addedHandler As Boolean
Private Shared addedHandlerLockObject As New Object
<Global.System.Diagnostics.DebuggerNonUserCodeAttribute(), Global.System.ComponentModel.EditorBrowsableAttribute(Global.System.ComponentModel.EditorBrowsableState.Advanced)> _
Private Shared Sub AutoSaveSettings(sender As Global.System.Object, e As Global.System.EventArgs)
If My.Application.SaveMySettingsOnExit Then
My.Settings.Save()
End If
End Sub
#End If
#End Region
Public Shared ReadOnly Property [Default]() As MySettings
Get
#If _MyType = "WindowsForms" Then
If Not addedHandler Then
SyncLock addedHandlerLockObject
If Not addedHandler Then
AddHandler My.Application.Shutdown, AddressOf AutoSaveSettings
addedHandler = True
End If
End SyncLock
End If
#End If
Return defaultInstance
End Get
End Property
End Class
End Namespace
Namespace My
<Global.Microsoft.VisualBasic.HideModuleNameAttribute(), _
Global.System.Diagnostics.DebuggerNonUserCodeAttribute(), _
Global.System.Runtime.CompilerServices.CompilerGeneratedAttribute()> _
Friend Module MySettingsProperty
<Global.System.ComponentModel.Design.HelpKeywordAttribute("My.Settings")> _
Friend ReadOnly Property Settings() As Global.XVIDEOS.My.MySettings
Get
Return Global.XVIDEOS.My.MySettings.Default
End Get
End Property
End Module
End Namespace

View File

@@ -0,0 +1,7 @@
<?xml version='1.0' encoding='utf-8'?>
<SettingsFile xmlns="http://schemas.microsoft.com/VisualStudio/2004/01/settings" CurrentProfile="(Default)" UseMySettingsClassName="true">
<Profiles>
<Profile Name="(Default)" />
</Profiles>
<Settings />
</SettingsFile>

View File

@@ -0,0 +1,175 @@
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="15.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Import Project="$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props" Condition="Exists('$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props')" />
<PropertyGroup>
<Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
<Platform Condition=" '$(Platform)' == '' ">AnyCPU</Platform>
<ProjectGuid>{CCCF47F4-C97C-4193-AC4B-C56DF2F9AA8A}</ProjectGuid>
<OutputType>Library</OutputType>
<RootNamespace>XVIDEOS</RootNamespace>
<AssemblyName>SCrawler.Plugin.XVIDEOS</AssemblyName>
<FileAlignment>512</FileAlignment>
<MyType>Windows</MyType>
<TargetFrameworkVersion>v4.6.1</TargetFrameworkVersion>
<Deterministic>true</Deterministic>
</PropertyGroup>
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
<DebugSymbols>true</DebugSymbols>
<DebugType>full</DebugType>
<DefineDebug>true</DefineDebug>
<DefineTrace>true</DefineTrace>
<OutputPath>bin\Debug\</OutputPath>
<DocumentationFile>
</DocumentationFile>
<NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
</PropertyGroup>
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
<DebugType>pdbonly</DebugType>
<DefineDebug>false</DefineDebug>
<DefineTrace>true</DefineTrace>
<Optimize>true</Optimize>
<OutputPath>bin\Release\</OutputPath>
<DocumentationFile>
</DocumentationFile>
<NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
</PropertyGroup>
<PropertyGroup>
<OptionExplicit>On</OptionExplicit>
</PropertyGroup>
<PropertyGroup>
<OptionCompare>Binary</OptionCompare>
</PropertyGroup>
<PropertyGroup>
<OptionStrict>Off</OptionStrict>
</PropertyGroup>
<PropertyGroup>
<OptionInfer>On</OptionInfer>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Debug|x64'">
<DebugSymbols>true</DebugSymbols>
<DefineDebug>true</DefineDebug>
<DefineTrace>true</DefineTrace>
<OutputPath>bin\x64\Debug\</OutputPath>
<NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
<DebugType>full</DebugType>
<PlatformTarget>x64</PlatformTarget>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Release|x64'">
<DefineTrace>true</DefineTrace>
<OutputPath>bin\x64\Release\</OutputPath>
<Optimize>true</Optimize>
<NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
<DebugType>pdbonly</DebugType>
<PlatformTarget>x64</PlatformTarget>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Debug|x86'">
<DebugSymbols>true</DebugSymbols>
<DefineDebug>true</DefineDebug>
<DefineTrace>true</DefineTrace>
<OutputPath>bin\x86\Debug\</OutputPath>
<NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
<DebugType>full</DebugType>
<PlatformTarget>x86</PlatformTarget>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Release|x86'">
<DefineTrace>true</DefineTrace>
<OutputPath>bin\x86\Release\</OutputPath>
<Optimize>true</Optimize>
<NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
<DebugType>pdbonly</DebugType>
<PlatformTarget>x86</PlatformTarget>
</PropertyGroup>
<ItemGroup>
<Reference Include="System" />
<Reference Include="System.Data" />
<Reference Include="System.Drawing" />
<Reference Include="System.Windows.Forms" />
<Reference Include="System.Xml" />
<Reference Include="System.Core" />
<Reference Include="System.Xml.Linq" />
<Reference Include="System.Data.DataSetExtensions" />
<Reference Include="System.Net.Http" />
</ItemGroup>
<ItemGroup>
<Import Include="Microsoft.VisualBasic" />
<Import Include="PersonalUtilities.Functions" />
<Import Include="SCrawler.Plugin" />
<Import Include="SCrawler.Plugin.Attributes" />
<Import Include="System" />
<Import Include="System.Collections" />
<Import Include="System.Collections.Generic" />
<Import Include="System.Data" />
<Import Include="System.Diagnostics" />
<Import Include="System.Linq" />
<Import Include="System.Xml.Linq" />
<Import Include="System.Threading.Tasks" />
</ItemGroup>
<ItemGroup>
<Compile Include="Declarations.vb" />
<Compile Include="M3U8.vb" />
<Compile Include="My Project\AssemblyInfo.vb" />
<Compile Include="My Project\Application.Designer.vb">
<AutoGen>True</AutoGen>
<DependentUpon>Application.myapp</DependentUpon>
<DesignTime>True</DesignTime>
</Compile>
<Compile Include="My Project\Resources.Designer.vb">
<AutoGen>True</AutoGen>
<DesignTime>True</DesignTime>
<DependentUpon>Resources.resx</DependentUpon>
</Compile>
<Compile Include="My Project\Settings.Designer.vb">
<AutoGen>True</AutoGen>
<DependentUpon>Settings.settings</DependentUpon>
<DesignTimeSharedInput>True</DesignTimeSharedInput>
</Compile>
<Compile Include="SettingsForm.Designer.vb">
<DependentUpon>SettingsForm.vb</DependentUpon>
</Compile>
<Compile Include="SettingsForm.vb">
<SubType>Form</SubType>
</Compile>
<Compile Include="SiteSettings.vb" />
<Compile Include="UserData.vb" />
</ItemGroup>
<ItemGroup>
<EmbeddedResource Include="My Project\Resources.resx">
<Generator>VbMyResourcesResXFileCodeGenerator</Generator>
<LastGenOutput>Resources.Designer.vb</LastGenOutput>
<CustomToolNamespace>My.Resources</CustomToolNamespace>
<SubType>Designer</SubType>
</EmbeddedResource>
<EmbeddedResource Include="SettingsForm.resx">
<DependentUpon>SettingsForm.vb</DependentUpon>
</EmbeddedResource>
</ItemGroup>
<ItemGroup>
<None Include=".editorconfig" />
<None Include="My Project\Application.myapp">
<Generator>MyApplicationCodeGenerator</Generator>
<LastGenOutput>Application.Designer.vb</LastGenOutput>
</None>
<None Include="My Project\Settings.settings">
<Generator>SettingsSingleFileGenerator</Generator>
<CustomToolNamespace>My</CustomToolNamespace>
<LastGenOutput>Settings.Designer.vb</LastGenOutput>
</None>
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\..\..\MyUtilities\PersonalUtilities\PersonalUtilities.vbproj">
<Project>{8405896b-2685-4916-bc93-1fb514c323a9}</Project>
<Name>PersonalUtilities</Name>
</ProjectReference>
<ProjectReference Include="..\SCrawler.PluginProvider\SCrawler.PluginProvider.vbproj">
<Project>{d4650f6b-5a54-44b6-999b-6c675b7116b1}</Project>
<Name>SCrawler.PluginProvider</Name>
</ProjectReference>
</ItemGroup>
<ItemGroup>
<None Include="Content\Pictures\IconPic32.png" />
</ItemGroup>
<ItemGroup>
<None Include="Content\Icons\Icon32.ico" />
</ItemGroup>
<Import Project="$(MSBuildToolsPath)\Microsoft.VisualBasic.targets" />
</Project>

View File

@@ -0,0 +1,129 @@
' Copyright (C) 2022 Andy
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
' (at your option) any later version.
'
' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY
<Global.Microsoft.VisualBasic.CompilerServices.DesignerGenerated()>
Partial Public Class SettingsForm : Inherits System.Windows.Forms.Form
<System.Diagnostics.DebuggerNonUserCode()>
Protected Overrides Sub Dispose(ByVal disposing As Boolean)
Try
If disposing AndAlso components IsNot Nothing Then
components.Dispose()
End If
Finally
MyBase.Dispose(disposing)
End Try
End Sub
Private components As System.ComponentModel.IContainer
<System.Diagnostics.DebuggerStepThrough()>
Private Sub InitializeComponent()
Dim CONTAINER_MAIN As System.Windows.Forms.ToolStripContainer
Dim resources As System.ComponentModel.ComponentResourceManager = New System.ComponentModel.ComponentResourceManager(GetType(SettingsForm))
Me.LIST_DOMAINS = New System.Windows.Forms.ListBox()
Me.ToolbarTOP = New System.Windows.Forms.ToolStrip()
Me.BTT_ADD = New System.Windows.Forms.ToolStripButton()
Me.BTT_DELETE = New System.Windows.Forms.ToolStripButton()
CONTAINER_MAIN = New System.Windows.Forms.ToolStripContainer()
CONTAINER_MAIN.ContentPanel.SuspendLayout()
CONTAINER_MAIN.TopToolStripPanel.SuspendLayout()
CONTAINER_MAIN.SuspendLayout()
Me.ToolbarTOP.SuspendLayout()
Me.SuspendLayout()
'
'CONTAINER_MAIN
'
'
'CONTAINER_MAIN.ContentPanel
'
CONTAINER_MAIN.ContentPanel.Controls.Add(Me.LIST_DOMAINS)
CONTAINER_MAIN.ContentPanel.Size = New System.Drawing.Size(384, 266)
CONTAINER_MAIN.Dock = System.Windows.Forms.DockStyle.Fill
CONTAINER_MAIN.LeftToolStripPanelVisible = False
CONTAINER_MAIN.Location = New System.Drawing.Point(0, 0)
CONTAINER_MAIN.Name = "CONTAINER_MAIN"
CONTAINER_MAIN.RightToolStripPanelVisible = False
CONTAINER_MAIN.Size = New System.Drawing.Size(384, 291)
CONTAINER_MAIN.TabIndex = 0
'
'CONTAINER_MAIN.TopToolStripPanel
'
CONTAINER_MAIN.TopToolStripPanel.Controls.Add(Me.ToolbarTOP)
'
'LIST_DOMAINS
'
Me.LIST_DOMAINS.Dock = System.Windows.Forms.DockStyle.Fill
Me.LIST_DOMAINS.FormattingEnabled = True
Me.LIST_DOMAINS.Location = New System.Drawing.Point(0, 0)
Me.LIST_DOMAINS.Name = "LIST_DOMAINS"
Me.LIST_DOMAINS.Size = New System.Drawing.Size(384, 266)
Me.LIST_DOMAINS.TabIndex = 0
'
'ToolbarTOP
'
Me.ToolbarTOP.Dock = System.Windows.Forms.DockStyle.None
Me.ToolbarTOP.GripStyle = System.Windows.Forms.ToolStripGripStyle.Hidden
Me.ToolbarTOP.Items.AddRange(New System.Windows.Forms.ToolStripItem() {Me.BTT_ADD, Me.BTT_DELETE})
Me.ToolbarTOP.Location = New System.Drawing.Point(0, 0)
Me.ToolbarTOP.Name = "ToolbarTOP"
Me.ToolbarTOP.Size = New System.Drawing.Size(384, 25)
Me.ToolbarTOP.Stretch = True
Me.ToolbarTOP.TabIndex = 0
'
'BTT_ADD
'
Me.BTT_ADD.AutoToolTip = False
Me.BTT_ADD.BackColor = System.Drawing.Color.FromArgb(CType(CType(192, Byte), Integer), CType(CType(255, Byte), Integer), CType(CType(192, Byte), Integer))
Me.BTT_ADD.ForeColor = System.Drawing.Color.DarkGreen
Me.BTT_ADD.Image = CType(resources.GetObject("BTT_ADD.Image"), System.Drawing.Image)
Me.BTT_ADD.ImageTransparentColor = System.Drawing.Color.Magenta
Me.BTT_ADD.Name = "BTT_ADD"
Me.BTT_ADD.Size = New System.Drawing.Size(49, 22)
Me.BTT_ADD.Text = "Add"
'
'BTT_DELETE
'
Me.BTT_DELETE.AutoToolTip = False
Me.BTT_DELETE.BackColor = System.Drawing.Color.FromArgb(CType(CType(255, Byte), Integer), CType(CType(192, Byte), Integer), CType(CType(192, Byte), Integer))
Me.BTT_DELETE.ForeColor = System.Drawing.Color.Maroon
Me.BTT_DELETE.Image = CType(resources.GetObject("BTT_DELETE.Image"), System.Drawing.Image)
Me.BTT_DELETE.ImageTransparentColor = System.Drawing.Color.Magenta
Me.BTT_DELETE.Name = "BTT_DELETE"
Me.BTT_DELETE.Size = New System.Drawing.Size(60, 22)
Me.BTT_DELETE.Text = "Delete"
'
'SettingsForm
'
Me.AutoScaleDimensions = New System.Drawing.SizeF(6.0!, 13.0!)
Me.AutoScaleMode = System.Windows.Forms.AutoScaleMode.Font
Me.ClientSize = New System.Drawing.Size(384, 291)
Me.Controls.Add(CONTAINER_MAIN)
Me.FormBorderStyle = System.Windows.Forms.FormBorderStyle.FixedSingle
Me.Icon = CType(resources.GetObject("$this.Icon"), System.Drawing.Icon)
Me.MaximizeBox = False
Me.MaximumSize = New System.Drawing.Size(400, 330)
Me.MinimizeBox = False
Me.MinimumSize = New System.Drawing.Size(400, 330)
Me.Name = "SettingsForm"
Me.SizeGripStyle = System.Windows.Forms.SizeGripStyle.Hide
Me.Text = "Settings"
CONTAINER_MAIN.ContentPanel.ResumeLayout(False)
CONTAINER_MAIN.TopToolStripPanel.ResumeLayout(False)
CONTAINER_MAIN.TopToolStripPanel.PerformLayout()
CONTAINER_MAIN.ResumeLayout(False)
CONTAINER_MAIN.PerformLayout()
Me.ToolbarTOP.ResumeLayout(False)
Me.ToolbarTOP.PerformLayout()
Me.ResumeLayout(False)
End Sub
Private WithEvents LIST_DOMAINS As Windows.Forms.ListBox
Private WithEvents ToolbarTOP As Windows.Forms.ToolStrip
Private WithEvents BTT_ADD As Windows.Forms.ToolStripButton
Private WithEvents BTT_DELETE As Windows.Forms.ToolStripButton
End Class

View File

@@ -0,0 +1,477 @@
<?xml version="1.0" encoding="utf-8"?>
<root>
<!--
Microsoft ResX Schema
Version 2.0
The primary goals of this format is to allow a simple XML format
that is mostly human readable. The generation and parsing of the
various data types are done through the TypeConverter classes
associated with the data types.
Example:
... ado.net/XML headers & schema ...
<resheader name="resmimetype">text/microsoft-resx</resheader>
<resheader name="version">2.0</resheader>
<resheader name="reader">System.Resources.ResXResourceReader, System.Windows.Forms, ...</resheader>
<resheader name="writer">System.Resources.ResXResourceWriter, System.Windows.Forms, ...</resheader>
<data name="Name1"><value>this is my long string</value><comment>this is a comment</comment></data>
<data name="Color1" type="System.Drawing.Color, System.Drawing">Blue</data>
<data name="Bitmap1" mimetype="application/x-microsoft.net.object.binary.base64">
<value>[base64 mime encoded serialized .NET Framework object]</value>
</data>
<data name="Icon1" type="System.Drawing.Icon, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
<value>[base64 mime encoded string representing a byte array form of the .NET Framework object]</value>
<comment>This is a comment</comment>
</data>
There are any number of "resheader" rows that contain simple
name/value pairs.
Each data row contains a name, and value. The row also contains a
type or mimetype. Type corresponds to a .NET class that support
text/value conversion through the TypeConverter architecture.
Classes that don't support this are serialized and stored with the
mimetype set.
The mimetype is used for serialized objects, and tells the
ResXResourceReader how to depersist the object. This is currently not
extensible. For a given mimetype the value must be set accordingly:
Note - application/x-microsoft.net.object.binary.base64 is the format
that the ResXResourceWriter will generate, however the reader can
read any of the formats listed below.
mimetype: application/x-microsoft.net.object.binary.base64
value : The object must be serialized with
: System.Runtime.Serialization.Formatters.Binary.BinaryFormatter
: and then encoded with base64 encoding.
mimetype: application/x-microsoft.net.object.soap.base64
value : The object must be serialized with
: System.Runtime.Serialization.Formatters.Soap.SoapFormatter
: and then encoded with base64 encoding.
mimetype: application/x-microsoft.net.object.bytearray.base64
value : The object must be serialized into a byte array
: using a System.ComponentModel.TypeConverter
: and then encoded with base64 encoding.
-->
<xsd:schema id="root" xmlns="" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:msdata="urn:schemas-microsoft-com:xml-msdata">
<xsd:import namespace="http://www.w3.org/XML/1998/namespace" />
<xsd:element name="root" msdata:IsDataSet="true">
<xsd:complexType>
<xsd:choice maxOccurs="unbounded">
<xsd:element name="metadata">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="value" type="xsd:string" minOccurs="0" />
</xsd:sequence>
<xsd:attribute name="name" use="required" type="xsd:string" />
<xsd:attribute name="type" type="xsd:string" />
<xsd:attribute name="mimetype" type="xsd:string" />
<xsd:attribute ref="xml:space" />
</xsd:complexType>
</xsd:element>
<xsd:element name="assembly">
<xsd:complexType>
<xsd:attribute name="alias" type="xsd:string" />
<xsd:attribute name="name" type="xsd:string" />
</xsd:complexType>
</xsd:element>
<xsd:element name="data">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="value" type="xsd:string" minOccurs="0" msdata:Ordinal="1" />
<xsd:element name="comment" type="xsd:string" minOccurs="0" msdata:Ordinal="2" />
</xsd:sequence>
<xsd:attribute name="name" type="xsd:string" use="required" msdata:Ordinal="1" />
<xsd:attribute name="type" type="xsd:string" msdata:Ordinal="3" />
<xsd:attribute name="mimetype" type="xsd:string" msdata:Ordinal="4" />
<xsd:attribute ref="xml:space" />
</xsd:complexType>
</xsd:element>
<xsd:element name="resheader">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="value" type="xsd:string" minOccurs="0" msdata:Ordinal="1" />
</xsd:sequence>
<xsd:attribute name="name" type="xsd:string" use="required" />
</xsd:complexType>
</xsd:element>
</xsd:choice>
</xsd:complexType>
</xsd:element>
</xsd:schema>
<resheader name="resmimetype">
<value>text/microsoft-resx</value>
</resheader>
<resheader name="version">
<value>2.0</value>
</resheader>
<resheader name="reader">
<value>System.Resources.ResXResourceReader, System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
</resheader>
<resheader name="writer">
<value>System.Resources.ResXResourceWriter, System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
</resheader>
<metadata name="CONTAINER_MAIN.GenerateMember" type="System.Boolean, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
<value>False</value>
</metadata>
<metadata name="ToolbarTOP.TrayLocation" type="System.Drawing.Point, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
<value>17, 17</value>
</metadata>
<assembly alias="System.Drawing" name="System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
<data name="BTT_ADD.Image" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
<value>
iVBORw0KGgoAAAANSUhEUgAAABgAAAAYCAYAAADgdz34AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8
YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAAN+SURBVEhLrZVZSFRRGMdvKa6oE5Y1bpTtRY2pJYk3ayyd
xgrvaDXlQotLKVqSo7QR2QIREUX0EgUSET1V9NTyYBHt6ZTOqNPY7DomWNbriX/fuYzkg2jY/eAPM5zD
//+d7/7OvcJkFXIlqDX0ahDGE18LbJt6cSOtWWTaDhGbPonQdZEsIrY7cuWQwLapFzfZ9FlkES8ERL8W
EPteQMLHYFR69coFUNcs5o2AWe+nYW5HKFK7otHkL1IuoKBbZLzzFHMo0qwxWN8zG2eHSpULkGxalmIO
w2qrCvm2eJr/AlwZrlIuYN9XHcuwqKC3JWC3czGqXKm4OVKvXMARTyHTfUlEmWspatxp9D8Ld0ZM/xYw
EeejMnkMv0tdS1DnzkBzv4jT/jy0fm/EzLvhE0p9P7JV7jB3lHHiu6BHhMGmRbkjH3WebTjq245z/nL6
nYFjvnVo8efjzKAOrcONuD1skoNuDNfj8lAFWgZL0OQrphEWoMa1E+oHkZADuHn0K2L8rYDEjhBoCENO
itSXgr2u5TjkycTx/g1koMP5Qb0ccHogj8xEVLlTsaNvIXJ750BjiUZSewhU5LPTIv0N4J1zc76osUTR
ZjWKAuaHvZk44dPijF9HAXk4OaCFyZdNzyIde5zLUGifh5yeOKzoikJ8RzBU7wSE06Us+jQmoMAqIpnM
U6mDXJtaxrDSrUGDdw2O9a/HCTI96suByZuNw57VqKa1MtcybLXPxTo6qWzeHgx+Gbn5tCcCpHdjAnbZ
NmKtNZYwTESJYwnq6WE2EinNZMhNj3izZDXQqKppJJymLfZkZHfHYWlnJNQfghBFIw5rEyA8Jj2ggJdj
Ag769cw0ZGAtP8rYhZH97NLPanbpVxW7/PMAu/atlnSQzNeg0qWB0bkIensSmj3lqHUaf1d4ilipU2LG
LxIzWCUmfSZ9JL2VmBzAMR0PsVHxTbeGmlmleyWMjoXYTKfMpNPWuI3y2oTimE5WfOP1bw1sBz2X/N4E
pNONnm8Oxx6HQTYJbJt6cZOLAzUsrzdeftHxt2kskbLbFpjx/xY3OeWtYKssMUgkUmYQzvzbUDzK+f8W
N6l3lbCk9lCoCMOI5wKCnhIlZgUD9tkNjJuHUefTOYaPKOCNggEl3RILp84FukDCQ9I9CnihYEBxpySP
hNiWL5DURnqmVACxzI3G1aScC8If1IGynvf45pQAAAAASUVORK5CYII=
</value>
</data>
<data name="BTT_DELETE.Image" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
<value>
iVBORw0KGgoAAAANSUhEUgAAABgAAAAYCAYAAADgdz34AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8
YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAAVFSURBVEhLjZVrTJNXGMcLQmdHO6Sdt6ygAyYgN+1bS+uE
4ASlxVfSUkFkRMyouILinShGYzTRKCZL9mH7sH3YMj+YzUSXZXNeQOcEFkFhysbVQWlL6dsLvMwtWaBn
/1OKl1A2nuSX8p7znP/z5znnPa+ARntubkRjXNxnd5Ys6b2xbNmn+XL5WxgO80/+T/SoVOo/YmLu9S9d
+uhqfLwRQxEgxD9J44nZLH6Qm/tNT20tGbh4kTw9dMh3Q6NpyJBI3sF0+FRW8OhimHX2kpJB/vx5Ml5f
TwaKitwfJyZWxURGRmF6qkhDXNwn3QcOkKenTpGBkyeJFcmWI0fIdbX6XppEsgIpQYv0aTTvQdzCX7hA
eKzl6+rI+IkTpJdlPd/K5R8hRQrCBHdkMisVh3MyeOwYsSJp+OxZYq2t9d1QqX5OlUgSkPhKEbRlo724
2EqdU/FxiPOHD5NxGPOWlZHGxYs7kaYGEsGt6OhLvRUVZAAFLMB69CixY8HImTPEjnbdVqubU8TiRCT7
i/QoFBuHi4qsY+fOER5mxmGKx7rx/fvJ6O7dpEGl4splsq+RqgNSgWH58mXXlcp7/Tt3+oaQZAPDcDOC
PXGhZc6DB8kdFFktFic9ZJhcm9FoHzt92t8SHo55tJevqSFjEL+rUrkTRKIvIWwCaYBuuCCckckSUaTZ
smMHse/ZQxzV1WRk3z7CoZiH/jfV1b4WjaZjsKDAyh8/7nfMY45HLl9VNSWuVLqTRKJL0KsAyQHxUOAP
4WqpNOlHhaJpqLTU56isJE7gMpuJGwJeFPPu3UvG4HaMCuOZx7NfHO1t0mg86WIxFafOV4L54MVRDYRQ
JZWuvJWe3mzfupU4y8uJC3hMJuKFw1EUHMUGjhYUEO/69cSr0RA3aMrI8K6WSC5j/S5AnYvADPHpEK6J
ikpuSEtrtrHshHvbNuLSaomLYQgnlxNOIiFceDjh5s0jI2Kx73ZsrEcdFUU3tBKkgNfBrOLT4W/Xg/T0
9mEIOqgYcAIq7CcsjPzOMH8bExO/R/6HIBXMSdwf9rIys33dOotDJJopDuiYNT5+opFl+5OiowuwJBLM
TdxrMh3kNmzgHHA/m7gNDAJLcjJ5VFjYuX3t2lVY+p/Xij9cJlMdt2mTyxERMUPcid7TdlHxAdAbGkqe
gO5Vq3zthYW/GdVq2qbZi3h27TrG5eW5gzl3ymRkKC/vH3tCgo+K9wTE20ATeMwwk+1GY+f22YrgKNZx
Wq3HIRb7hV9py6JFpEun+/N0VlZL2+bNw5akJPIYoq3gPvgJNIL2NWsmW43Gjg1pafSCfHHVuysqajmd
zoNjN1N84ULSk5//rIphbiG1Oi8l5f1Ovb6/C72fFm8AN8F10KpSTbYUFrZ9wbJ044UgROA2GKwjCxYE
dU7FKxWKm0g0A7qR0n3Z2e92GAx9Hampz8V/AN+Ba6A9K2ui1WDoRu5yIBQ4jcYud0xMUOdmhrmNpKqA
+PSXSrg/J0eNje1rS031vSx+JSSEtGZmTtxlWRvyckGUoKW0tGSYZV0uiL7k/K89wcWnQ1iTnZ2Bje3F
C0muBsRxbUxe1mqHdCtWfI6cqQKI+TeLio5bt2zxONVqX49O96xGqWzA+Gzi0yE0ZWYqaDtase4XjWby
ilZre0MkqsecHky1CBHCxMZGPjQYvnqq1zsv5+Tg9IXWYJwBYjpPk2YJ4bXi4g9+1ett91mW0yYk0G+B
AcgBPa7P19I/FgAlyA/8zvX1fw3Qo0lPzmbwNgj6ws0D1DH9WNNf+jyXoCbo/f9mAFowYEwg+Bc5Ntw7
FHW1qQAAAABJRU5ErkJggg==
</value>
</data>
<data name="$this.Icon" type="System.Drawing.Icon, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
<value>
AAABAAUAEBAAAAEAIABoBAAAVgAAABgYAAABACAAiAkAAL4EAAAgIAAAAQAIAKgIAABGDgAAMDAAAAEA
IACoJQAA7hYAACAgAAABAAgAqAgAAJY8AAAoAAAAEAAAACAAAAABACAAAAAAAAAEAAAAAAAAAAAAAAAA
AAAAAAAAAAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAIO/wADF/8AAxb/AAEJ/wAAAP8AAAD/AAAA/wAAAP8AAQb/AAMV/wAD
F/8AAhL/AAAC/wAAAP8AAAD/AAAA/wAHMv8AFoz/ABiW/wAQaf8AAQz/AAAA/wAAAP8AAAT/AAxU/wAY
lP8AGJP/AAtJ/wAAAv8AAAD/AAAA/wAAAP8AAQ3/ABOQ/wAh3v8AH9T/AAtb/wAAAf8AAAD/AAc7/wAc
x/8AIeD/ABis/wADHv8AAAD/AAAA/wAAAP8AAAD/AAAA/wAFK/8AG73/ACLk/wAbvf8ABSr/AAIV/wAW
o/8AIuP/AB7P/wAIRv8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAH/AAtZ/wAf1v8AIeH/ABSU/wAQ
e/8AINz/ACDe/wAPdv8AAAX/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wABC/8AE4z/ACHg/wAg
2/8AINn/ACHi/wAXpf8AAhn/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAQm/wAa
t/8AIeH/ACHh/wAdyP8ABzr/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAB
C/8AFZj/ACHi/wAh4f8AG7r/AAQh/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AClH/AB7S/wAh4P8AIeD/ACHe/wAQfP8AAAb/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8ABCX/ABq2/wAi4/8AGrn/ABag/wAh4P8AHtD/AAlH/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAQr/ABKI/wAh4f8AH9X/AApQ/wAEKP8AG7v/ACLk/wAYrv8AAx7/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/Ag1W/wIi1/8CJOP/ABOL/wABCv8AAAD/AAxc/wIi2f8CJOL/AhN//wABB/8AAAD/AAAA/wAA
AP8AAAD/CQ0h/ylBsv8vTt7/Iz67/wMJKv8AAAD/AAAA/wACEP8aMJr/L07e/y1Iyv8RGD//AAAA/wAA
AP8AAAD/AAAA/wUHEP8RFzL/Ehg0/wwQI/8BAQL/AAAA/wAAAP8AAAD/CAsY/xIYM/8SGDT/CAsZ/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAoAAAAGAAAADAAAAABACAAAAAAAAAJAAAAAAAAAAAAAAAAAAAAAAAAAAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAEM/wADG/8AAxr/AAMa/wADFv8AAQT/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAB/wACEf8AAxv/AAMa/wADG/8AAhT/AAAC/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAQk/wAT
dv8AF4b/ABeF/wAVgP8ACT7/AAAC/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAQl/wATdP8AF4X/ABeF/wAW
gf8ACUD/AAAD/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAI/wAOav8AHcX/AB/K/wAfyv8AGaz/AAQv/wAA
AP8AAAD/AAAA/wAAAP8AART/ABKL/wAeyf8AH8r/AB/L/wAVkv8AAhr/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wADJP8AF6v/ACHi/wAh4P8AIeD/ABKR/wABE/8AAAD/AAAA/wAAA/8ADGH/AB/W/wAh
4f8AIuL/ABzJ/wAISf8AAAH/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AB0b/AB3K/wAh
4P8AIeD/AB7U/wALYP8AAAT/AAAA/wAFN/8AGr3/ACHi/wAh4P8AINr/AA5y/wAAB/8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAf/AA53/wAg2v8AIeD/ACHh/wAauv8ABTL/AAIT/wAT
k/8AId//ACHf/wAh4P8AFaD/AAIb/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAIg/wAVpP8AIeH/ACHg/wAh3v8AE5X/AA1z/wAf1P8AIeD/ACHh/wAbwf8ABjz/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAHQf8AHMX/ACHh/wAh
3/8AINr/AB/W/wAh4P8AIeD/AB/W/wAMZ/8AAAb/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAACP8ADXD/ACDZ/wAh4P8AId//ACHg/wAh3/8AId//ABSU/wAB
Fv8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAhj/ABWe/wAh3/8AId//ACHf/wAh4P8AGrz/AAUx/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAT/AAxr/wAg3P8AId//ACHf/wAh
4f8AFqP/AAIT/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAyX/ABiv/wAh4P8AId//ACHf/wAh4P8AH9L/AApb/wAAAv8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wABDf8AEYL/ACDd/wAh
3/8AIeD/ACHg/wAh3/8AIeH/ABm3/wAFL/8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAC/wAKU/8AHs7/ACHh/wAh4P8AHMX/ABeo/wAg3P8AIeD/ACHf/wAS
jv8AARL/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAQq/wAY
sP8AIeH/ACHg/wAf1/8ADGj/AAQy/wAYtP8AIeH/ACHg/wAe0v8AC1v/AAAD/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAQv/ABGG/wAg3f8AIeD/ACHh/wAVnf8AAhr/AAAB/wAL
XP8AH9P/ACHg/wAh4P8AGrn/AAQu/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
Av8ACVX/AB3R/wAg4P8AIOH/ABvE/wAHPv8AAAD/AAAA/wABE/8AE5L/ACDf/wAg3/8AIN//ABGO/wAB
Ef8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wULLv8NKbj/DjHn/w4x5f8MLt//ARFz/wAA
CP8AAAD/AAAA/wAAAP8ABTT/ByXD/w4x5v8OMeb/Di7Y/wgWYf8BAQT/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/BAYN/yg5hf9BXuH/QF7i/0Ff4/8tRa7/Awgd/wAAAP8AAAD/AAAA/wAAAP8AAQX/GChx/z9b
3f9BX+L/QV/k/zZOt/8MECj/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AgMJ/w8VMf8VHT//FRw9/xUc
Pv8KDiH/AAEB/wAAAP8AAAD/AAAA/wAAAP8AAAD/BQcP/xMZOP8VHD7/FRw+/xMbPP8FBxL/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAKAAAACAA
AABAAAAAAQAIAAAAAAAABAAAAAAAAAAAAAAAAQAAAAEAAAAAAAAIDigAFBxBABMbPwADBhQADRMxABQb
QAASGj0AAQEGABsnWgBSbewAVXHyAFRx8QBVcvQANUqlAAABAwANFz8ATmjiAFVy8gA+U7UAAQIKAAAA
AgAULJcAG0LsABtB6gAZP+QAAQ1bAAAACAALKawAHELtABtB6wAbPtgABg46AAABHgAAGb0AACDhAAAg
3wAAIeEAABq/AAABIAAACGcAACDeAAAKcAAABlEAACDYAAAh4AAAId8AACHiAAARjQAAAAEAAAIpAAAc
xwAAIuIAABWoAAAACgAAEIsAAAZPAAAAAwAAE5cAAB7OAAADNgAAARsAABq7AAAZugAAARkAAAZXAAAg
2wAAC3AAAAVNAAAg1wAADoMAAAEiAAAavQAAEIcAAB/TAAAYtAAAARgAABm4AAAFSQAAH9YAAA6CAAAe
zAAAAzUAAAIzAAAQiAAAEIkAAAZSAAAg2gAAEpIAACDcAAAbwgAAASUAAAZbAAATmwAAB2EAABOZAAAA
BQAAAScAABvEAAACKgAAEY4AACHdAAAIZAAAAAQAAAhjAAABHwAAGr4AABzKAAACMQAACGIAACHcAAAT
nAAABlMAACDZAAAVpgAAAi0AABzIAAALcgAAAAYAABSgAAAe0AAAAzwAAAdOAAAcsQAAHLMAAB61AAAR
gQAABUcAABuvAAActAAAE4cAAAEVAAAUdQAAGIEAABeAAAAXgQAAE3MAAAtTAAAHQwAAAAcAAAIaAAAC
GQAAAQ4AAAEUAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAi4yNjY2MjgAAAAAAAAAAAAA5jY2NjYyPAAAAAAAAAACDhIWGhoeIgwAA
AAAAAAAAMYmFhoaGhYoAAAAAAAAAAAB6e3x8fH1+ZwAAAAAAAAB/gIF8fH2CGwAAAAAAAAAAAHZ3Ly4u
LlleAAAAAAAAaUglLi4teHkAAAAAAAAAAAAAAHN0JS4uJTNjAAAAABVkLy4uLi51AAAAAAAAAAAAAAAA
AG1uLi4uL29gAAAAcHEtLi40cjYAAAAAAAAAAAAAAAAAZ18vLi4uZWgAAGlqJS4uJWtsAAAAAAAAAAAA
AAAAAAAAYWIlLi4lM2MxZC8uLi5lZgAAAAAAAAAAAAAAAAAAAAAAXEItLi40XV4sLS4uL19gAAAAAAAA
AAAAAAAAAAAAAAAVWC8uLi5ZTy0uLiVaWwAAAAAAAAAAAAAAAAAAAAAAAAAnJiUuLi4tLi4tV0EAAAAA
AAAAAAAAAAAAAAAAAAAAAABWLC0uLi4uLiUwFQAAAAAAAAAAAAAAAAAAAAAAAAAAADFVLS4uLi4lTT0A
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAFNKLS4uLjRUAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAxUC4uLi4u
LVFSAAAAAAAAAAAAAAAAAAAAAAAAAAAAAE5PLS4uLi4uNDU2AAAAAAAAAAAAAAAAAAAAAAAAAABMTS8u
Li0lLi4uLkMAAAAAAAAAAAAAAAAAAAAAAAAAMUklLi4tSksuLi4tOzwAAAAAAAAAAAAAAAAAAAAAAABE
RS0uLiVGR0glLi40NTYAAAAAAAAAAAAAAAAAAAAAPT4lLi4lP0AAQUIuLi4uQwAAAAAAAAAAAAAAAAAA
ADE3JS4uLSw4AAA5OjQuLi07PAAAAAAAAAAAAAAAAAAAKywtLi4vMDEAAAAyMyUuLjQ1NgAAAAAAAAAA
AAAAACEiIyQkJSYnAAAAAAAoKSQkJCQqAAAAAAAAAAAAAAAVFhcYGBgZGgAAAAAAABscHRgYHh8gAAAA
AAAAAAAAAAkKCwwMDQ4PAAAAAAAAABAREgwMDRMUAAAAAAAAAAAAAQIDAwMDBAAAAAAAAAAAAAUGAwMD
BwgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAoAAAAMAAAAGAA
AAABACAAAAAAAAAkAAAAAAAAAAAAAAAAAAAAAAAAAAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAC/wAAB/8AAAr/AAAJ/wAA
Cf8AAAn/AAAJ/wAACf8AAAj/AAAF/wAAAv8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAH/AAAH/wAACf8AAAn/AAAJ/wAACf8AAAn/AAAJ/wAA
Cf8AAAf/AAAB/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAH/AAAI/wAD
Gv8ABSr/AAYq/wAGKv8ABSr/AAUq/wAGK/8ABSn/AAQh/wACEP8AAAL/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wABB/8AAhT/AAUm/wAGK/8ABSr/AAUq/wAF
Kv8ABir/AAYr/wAEJf8AAhj/AAAF/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAP/AAEP/wAJPf8AEmj/ABRv/wAUb/8AE2//ABNv/wAUcP8AE27/ABFl/wAIOf8AAQ3/AAAC/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAEIf8AC03/ABJp/wAU
cP8AE2//ABNv/wATb/8AFG//ABRw/wANWP8ABTH/AAEK/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAH/AAAJ/wAHOP8AEXH/ABmV/wAbnv8AGpv/ABqc/wAanP8AGpz/ABmY/wAR
cv8ABzz/AAEM/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAX/AAEa/wAK
Uf8AFYf/ABmZ/wAbnf8AGpz/ABqc/wAbnf8AG5v/ABiO/wANWv8AAyD/AAAG/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wACHf8ACVT/ABed/wAduf8AHbv/AB27/wAd
u/8AHbv/AB68/wAaq/8AEHz/AAQo/wAABf8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AARL/AAdM/wATjP8AHLn/AB28/wAdu/8AHbv/AB27/wAevf8AHLH/ABOG/wAIP/8AAAT/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAABP8ABCr/ABKL/wAc
xP8AINv/ACDZ/wAg2P8AINj/ACHY/wAg1/8AGrn/AApm/wACJP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAACP8ABDH/ABOR/wAcxv8AINn/ACDZ/wAg2P8AINj/ACDZ/wAg1/8AGrf/AApf/wAC
IP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAhH/AAlV/wAVpP8AINz/ACHh/wAh4f8AIeH/ACHg/wAi5P8AH9b/ABSk/wAIUv8AAA3/AAAC/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAEJv8ADGv/ABu//wAg3/8AIeL/ACHh/wAh4P8AIeH/ACLj/wAc
yf8AEY7/AAQy/wAACf8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAX/AAEe/wANbv8AGr//ACDb/wAh4f8AId//ACHf/wAh4f8AId7/AB7R/wAR
hv8ABDL/AAEJ/wAAAP8AAAD/AAAA/wAAAP8AAAH/AAEL/wAKU/8AFqj/AB/X/wAh4/8AId//ACHf/wAh
4P8AId//ACDZ/wAUnv8AB0//AAEP/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAF/wAFOv8AD4b/AB7M/wAi4v8AId//ACHf/wAh
3/8AIeD/ACHf/wAZuP8ADXL/AAMZ/wAAAP8AAAD/AAAA/wAAAP8AAQv/AAQ4/wARi/8AHtH/ACHe/wAh
4P8AId//ACHf/wAi4v8AINv/ABm6/wAMZf8AARX/AAAD/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wABE/8AB0j/ABem/wAf
1f8AIeH/ACHg/wAh3/8AIeD/ACHh/wAf1v8AF6j/AAdJ/wABE/8AAAD/AAAA/wAAAf8AAyD/AA10/wAZ
uf8AIeD/ACHg/wAh3/8AId//ACHg/wAh4f8AHMj/AA9+/wAFM/8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAxj/AA1v/wAZtv8AIN//ACHg/wAh3/8AId//ACHf/wAi4v8AHs3/AA+H/wAFOv8AAAT/AAAA/wAB
E/8ACEr/ABes/wAf2P8AIeH/ACHf/wAh3/8AIeD/ACHh/wAe0v8AFaH/AAVA/wAAD/8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAn/AAMv/wAQg/8AHc//ACDe/wAh4f8AId//ACHf/wAh4f8AINv/ABq//wAN
bP8AARv/AAAE/wAGOf8AEYz/AB3N/wAh4f8AId//ACHf/wAh3/8AIeD/ACHf/wAXrv8ACWL/AAIZ/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAL/AAAM/wAITv8AE6D/AB7U/wAi4/8AId//ACHf/wAh
4P8AIeD/ACDa/wAVnv8ACVf/AAQ0/wANbv8AG7//ACDc/wAh4v8AId//ACHf/wAh4f8AIN3/ABzG/wAP
ef8AAyj/AAAH/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wACIf8ACWL/ABq6/wAg
3f8AIeD/ACHf/wAh3/8AIeD/ACLi/wAcxv8AE53/AA18/wAVpv8AIdv/ACHg/wAh4P8AId//ACHf/wAh
4f8AHtD/ABOV/wAHQv8AAAL/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
A/8ABCj/ABGH/wAbxf8AIeL/ACHg/wAh3/8AId//ACHg/wAg2f8AHc//ABvF/wAe0f8AId//ACHf/wAh
3/8AId//ACHh/wAf2f8AGLD/AAhQ/wABF/8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAQ7/AAdI/wAUmf8AH9f/ACHf/wAh4P8AId//ACHf/wAh3/8AIeD/ACHf/wAh
3/8AIeD/ACHf/wAh3/8AIeD/ACHh/wAZvP8ADnj/AAMj/wAAA/8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAP/AAAV/wALY/8AGbf/AB/Z/wAh4f8AId//ACHf/wAh
3/8AId//ACHg/wAh3/8AId//ACHf/wAh4P8AId7/AB7S/wARjf8ABDv/AAEL/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAD/wAEMf8ADXn/ABzH/wAh
4f8AIeD/ACHf/wAh3/8AId//ACHf/wAh3/8AId//ACHf/wAh4v8AH9b/ABan/wAKU/8AAQz/AAAC/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
Dv8ABj3/ABWb/wAez/8AIeH/ACHg/wAh3/8AId//ACHf/wAh3/8AId//ACHg/wAg2/8AGrr/AAxn/wAE
Jf8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAhT/AAtl/wAYsP8AId//ACHg/wAh3/8AId//ACHf/wAh3/8AIeD/ACHh/wAd
zv8AE5X/AAQv/wAABf8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAr/AAM1/wASjf8AINj/ACHf/wAh4P8AId//ACHf/wAh
3/8AIeD/ACLi/wAavv8ADnf/AAMa/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAQ3/AAZG/wAUmf8AINv/ACHf/wAh
4P8AId//ACHf/wAh3/8AIeD/ACLh/wAdyf8AE4//AAQu/wAABv8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAv8AAyL/AA56/wAa
vP8AIeD/ACHg/wAh3/8AId//ACHf/wAh3/8AId//ACHg/wAh3f8AG7r/AApf/wACIP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAB
Gf8ACFT/ABix/wAf2f8AIeD/ACHf/wAh3/8AId//ACHf/wAh3/8AId//ACHf/wAi4v8AH9P/ABWi/wAI
TP8AAAX/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAG/wAHQv8AEpL/AB7Q/wAh4/8AId//ACHf/wAh3/8AIeD/ACHh/wAh4P8AId//ACHf/wAi
4f8AId7/AB3N/wAQgv8ABC//AAEJ/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAb/AAMi/wAPdf8AHMb/ACDd/wAh4f8AId//ACHf/wAh3/8AId7/ACDb/wAg
3f8AId//ACHf/wAh3/8AIeD/ACHg/wAYtP8ADGz/AAMc/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAhP/AAtg/wAXq/8AINz/ACHg/wAh3/8AId//ACHg/wAg
3P8AHtH/ABvA/wAdzf8AId//ACHf/wAh3/8AId//ACHg/wAf1f8AF6b/AAZF/wAAEf8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAADf8ABjv/ABWa/wAez/8AIeL/ACHg/wAh
3/8AIeD/ACHi/wAcy/8AFaL/AA51/wAVm/8AH9X/ACHf/wAh4P8AId//ACHf/wAi4f8AHcr/AA+C/wAF
NP8AAAH/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAC/wADLv8ADXX/ABzF/wAh
4f8AIeD/ACHf/wAh3/8AIeD/ACDd/wAWqf8ACmH/AAIo/wALX/8AGbb/AB/Z/wAh4v8AId//ACHf/wAi
4v8AINv/ABm7/wAMZf8AARX/AAAD/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAP/AAAR/wAL
Xv8AGLP/AB/Y/wAh4f8AId//ACHf/wAh4f8AIN3/ABzI/wAPdv8AAiL/AAAJ/wAEMf8ADXz/ABzK/wAh
4v8AId//ACHf/wAh4P8AId//ACDZ/wAUnf8AB07/AAEP/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAQ3/AAZD/wATlP8AH9X/ACHf/wAh4P8AId//ACHf/wAi4v8AHtH/ABKW/wAHRf8AAAb/AAAA/wAA
D/8ABkL/ABak/wAf1P8AIuH/ACHg/wAh3/8AIeD/ACHh/wAcx/8AEYr/AAQu/wAAB/8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAv8ABCT/ABCB/wAbwv8AIeH/ACHg/wAh3/8AId//ACHg/wAg2/8AGbX/AAlZ/wAC
HP8AAAD/AAAA/wAAAP8AAxf/AA1u/wAZtv8AId//ACHg/wAh3/8AId//ACHg/wAh3v8AG7v/AApg/wAC
IP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wACHf8ACVr/ABm1/wAg2/8AIeD/ACHf/wAh3/8AIeD/ACHi/wAb
w/8AEIL/AAQk/wAAAv8AAAD/AAAA/wAAAP8AAQn/AAQw/wARhf8AHtH/ACHe/wAh4f8AId//ACHf/wAi
4v8AH9T/ABWi/wAITP8AAAX/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAH/AAAI/wAHR/8AEZf/AB3R/wAg4v8AH9//AB/f/wAg
4P8AIN//AB7X/wATl/8AB0b/AAEN/wAAAP8AAAD/AAAA/wAAAP8AAAL/AAAN/wAJU/8AFab/AB7W/wAg
4v8AH97/AB/f/wAg4f8AH97/ABvM/wAOgf8AAy//AAAJ/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAQf/AgYp/wQUfP8EIcv/BCTf/wQl
4/8EJeH/BCXh/wQm4/8DJNr/ARq4/wAMY/8AABX/AAAD/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAC
J/8AC2z/Ah3C/wMk4P8EJeH/BCXh/wQl4f8EJeL/BCXi/wQctv8EEG3/AQQc/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8DBhb/DR5s/xQx
uP8WO+b/Fjvp/xY76P8WO+j/Fjvo/xY86v8RM9H/BBaB/wAGNP8AAAP/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAABv8BBzH/CCGa/xAz1f8XPOv/Fjvp/xY76P8WO+n/Fjvp/xc63v8TLq7/CBJJ/wIE
Ev8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wYI
Ev8VH0r/L0ey/zpZ5/87Xvf/O131/ztd9P87XfX/O172/zRV5f8jPa//BxFG/wABEP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8CBRP/Chti/yM+tv85WvD/O130/zte9f87XfT/O131/zxe
9v84Vt7/KDqS/xEZPv8AAQT/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/woPJf8gLmj/QFa8/0tk1/9KY9T/SmPU/0pj1P9KZNX/SmTW/z1Ttv8lNHX/BwsZ/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8BAwj/CQ8q/yc3ff9FXMj/SmPT/0pk
1f9KY9T/SmPU/0tk1/9IYM//O0+s/xslVP8AAQf/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wUIF/8PFzn/HCZX/x8rYP8fKl3/Hipd/x4qXf8fKl7/Hype/xYe
Rv8KDiT/AgMH/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAH/AgMI/w0T
LP8aI1L/Hilc/x8qXv8eKl3/Hipd/x8qXv8eKV3/GyVV/w0TLP8BAQX/AAAB/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wIDCf8EBxT/BwoZ/wgLGv8HCxn/BwsZ/wcL
Gf8HCxn/BwsZ/wQGD/8AAAT/AAAB/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wIDCf8FBxT/BwoZ/wgLGv8HCxn/BwsZ/wcKGf8HCxr/Bwsa/wQGD/8AAAL/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAAAP8AAAD/AAAA/wAA
AP8AAAD/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAKAAAACAAAABAAAAAAQAIAAAAAAAABAAAAAAAAAAA
AAAAAQAAAAEAAAAAAAAIDigAFBxBABMbPwADBhQADRMxABQbQAASGj0AAQEGABsnWgBSbewAVXHyAFRx
8QBVcvQANUqlAAABAwANFz8ATmjiAFVy8gA+U7UAAQIKAAAAAgAULJcAG0LsABtB6gAZP+QAAQ1bAAAA
CAALKawAHELtABtB6wAbPtgABg46AAABHgAAGb0AACDhAAAg3wAAIeEAABq/AAABIAAACGcAACDeAAAK
cAAABlEAACDYAAAh4AAAId8AACHiAAARjQAAAAEAAAIpAAAcxwAAIuIAABWoAAAACgAAEIsAAAZPAAAA
AwAAE5cAAB7OAAADNgAAARsAABq7AAAZugAAARkAAAZXAAAg2wAAC3AAAAVNAAAg1wAADoMAAAEiAAAa
vQAAEIcAAB/TAAAYtAAAARgAABm4AAAFSQAAH9YAAA6CAAAezAAAAzUAAAIzAAAQiAAAEIkAAAZSAAAg
2gAAEpIAACDcAAAbwgAAASUAAAZbAAATmwAAB2EAABOZAAAABQAAAScAABvEAAACKgAAEY4AACHdAAAI
ZAAAAAQAAAhjAAABHwAAGr4AABzKAAACMQAACGIAACHcAAATnAAABlMAACDZAAAVpgAAAi0AABzIAAAL
cgAAAAYAABSgAAAe0AAAAzwAAAdOAAAcsQAAHLMAAB61AAARgQAABUcAABuvAAActAAAE4cAAAEVAAAU
dQAAGIEAABeAAAAXgQAAE3MAAAtTAAAHQwAAAAcAAAIaAAACGQAAAQ4AAAEUAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAi4yNjY2M
jgAAAAAAAAAAAAA5jY2NjYyPAAAAAAAAAACDhIWGhoeIgwAAAAAAAAAAMYmFhoaGhYoAAAAAAAAAAAB6
e3x8fH1+ZwAAAAAAAAB/gIF8fH2CGwAAAAAAAAAAAHZ3Ly4uLlleAAAAAAAAaUglLi4teHkAAAAAAAAA
AAAAAHN0JS4uJTNjAAAAABVkLy4uLi51AAAAAAAAAAAAAAAAAG1uLi4uL29gAAAAcHEtLi40cjYAAAAA
AAAAAAAAAAAAZ18vLi4uZWgAAGlqJS4uJWtsAAAAAAAAAAAAAAAAAAAAYWIlLi4lM2MxZC8uLi5lZgAA
AAAAAAAAAAAAAAAAAAAAXEItLi40XV4sLS4uL19gAAAAAAAAAAAAAAAAAAAAAAAVWC8uLi5ZTy0uLiVa
WwAAAAAAAAAAAAAAAAAAAAAAAAAnJiUuLi4tLi4tV0EAAAAAAAAAAAAAAAAAAAAAAAAAAABWLC0uLi4u
LiUwFQAAAAAAAAAAAAAAAAAAAAAAAAAAADFVLS4uLi4lTT0AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAFNK
LS4uLjRUAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAxUC4uLi4uLVFSAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AE5PLS4uLi4uNDU2AAAAAAAAAAAAAAAAAAAAAAAAAABMTS8uLi0lLi4uLkMAAAAAAAAAAAAAAAAAAAAA
AAAAMUklLi4tSksuLi4tOzwAAAAAAAAAAAAAAAAAAAAAAABERS0uLiVGR0glLi40NTYAAAAAAAAAAAAA
AAAAAAAAPT4lLi4lP0AAQUIuLi4uQwAAAAAAAAAAAAAAAAAAADE3JS4uLSw4AAA5OjQuLi07PAAAAAAA
AAAAAAAAAAAAKywtLi4vMDEAAAAyMyUuLjQ1NgAAAAAAAAAAAAAAACEiIyQkJSYnAAAAAAAoKSQkJCQq
AAAAAAAAAAAAAAAVFhcYGBgZGgAAAAAAABscHRgYHh8gAAAAAAAAAAAAAAkKCwwMDQ4PAAAAAAAAABAR
EgwMDRMUAAAAAAAAAAAAAQIDAwMDBAAAAAAAAAAAAAUGAwMDBwgAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
</value>
</data>
</root>

View File

@@ -0,0 +1,73 @@
' Copyright (C) 2022 Andy
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
' (at your option) any later version.
'
' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY
Imports PersonalUtilities.Forms.Toolbars
Imports PersonalUtilities.Forms
Public Class SettingsForm : Implements IOkCancelToolbar
Private ReadOnly MyDefs As DefaultFormProps
Private ReadOnly Property Settings As SiteSettings
Friend Sub New(ByRef s As SiteSettings)
InitializeComponent()
MyDefs = New DefaultFormProps
Settings = s
End Sub
Private Sub SettingsForm_Load(sender As Object, e As EventArgs) Handles Me.Load
Try
With MyDefs
.MyViewInitialize(Me, Settings.Design, True)
.AddOkCancelToolbar()
.DelegateClosingChecker()
If Settings.Domains.Count > 0 Then Settings.Domains.ForEach(Sub(d) LIST_DOMAINS.Items.Add(d))
.EndLoaderOperations()
End With
Catch ex As Exception
MyDefs.InvokeLoaderError(ex)
End Try
End Sub
Private Sub ToolbarBttOK() Implements IOkCancelToolbar.ToolbarBttOK
Settings.Domains.Clear()
With LIST_DOMAINS
If .Items.Count > 0 Then
For Each i In .Items : Settings.Domains.Add(i.ToString) : Next
End If
End With
Settings.UpdateDomains()
MyDefs.CloseForm()
End Sub
Private Sub ToolbarBttCancel() Implements IOkCancelToolbar.ToolbarBttCancel
MyDefs.CloseForm(Windows.Forms.DialogResult.Cancel)
End Sub
Private Sub BTT_ADD_Click(sender As Object, e As EventArgs) Handles BTT_ADD.Click
Dim nd$ = InputBoxE("Enter a new domain using the pattern [xvideos.com]:", "New domain")
If Not nd.IsEmptyString Then
If Not LIST_DOMAINS.Items.Contains(nd) Then
LIST_DOMAINS.Items.Add(nd)
Else
MsgBoxE($"The domain [{nd}] already added")
End If
End If
End Sub
Private Sub BTT_DELETE_Click(sender As Object, e As EventArgs) Handles BTT_DELETE.Click
If _LatestSelected.ValueBetween(0, LIST_DOMAINS.Items.Count - 1) Then
Dim n$ = LIST_DOMAINS.Items(_LatestSelected)
If MsgBoxE({$"Are you sure you want to delete the [{n}] domain?",
"Removing domains"}, MsgBoxStyle.YesNo) = MsgBoxResult.Yes Then
LIST_DOMAINS.Items.RemoveAt(_LatestSelected)
MsgBoxE($"Domain [{n}] removed")
Else
MsgBoxE("Operation canceled")
End If
Else
MsgBoxE("No domain selected", vbExclamation)
End If
End Sub
Private _LatestSelected As Integer = -1
Private Sub LIST_DOMENS_SelectedIndexChanged(sender As Object, e As EventArgs) Handles LIST_DOMAINS.SelectedIndexChanged
_LatestSelected = LIST_DOMAINS.SelectedIndex
End Sub
End Class

View File

@@ -0,0 +1,185 @@
' Copyright (C) 2022 Andy
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
' (at your option) any later version.
'
' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY
Imports System.Drawing
Imports PersonalUtilities.Functions.XML
Imports PersonalUtilities.Functions.RegularExpressions
Imports PersonalUtilities.Tools.WEB
<Manifest("AndyProgram_XVIDEOS"), SpecialForm(True)>
Public Class SiteSettings : Implements ISiteSettings
Public ReadOnly Property Icon As Icon Implements ISiteSettings.Icon
Get
Return My.Resources.Icon32
End Get
End Property
Public ReadOnly Property Image As Image Implements ISiteSettings.Image
Get
Return My.Resources.IconPic32
End Get
End Property
Public ReadOnly Property Site As String = "XVIDEOS" Implements ISiteSettings.Site
Public Property Logger As ILogProvider Implements ISiteSettings.Logger
#Region "M3U8"
Private ReadOnly OS64 As Boolean
Private ReadOnly FfmpegExists As Boolean
Friend ReadOnly FfmpegFile As SFile
Friend ReadOnly Property UseM3U8 As Boolean
Get
Return OS64 And FfmpegExists
End Get
End Property
#End Region
<PXML("Domains")> Private Property SiteDomains As PropertyValue
<PropertyOption(ControlText:="Download UHD", ControlToolTip:="Download UHD (4K) content"), PXML>
Public Property DownloadUHD As PropertyValue
Friend ReadOnly Property Domains As List(Of String)
Public ReadOnly Property Responser As Response
Private Const DomainsDefault As String = "xvideos.com|xnxx.com"
Private _Initialized As Boolean = False
Friend Design As XmlFile
Public Sub New()
Responser = New Response($"Settings\Responser_{Site}.xml")
With Responser
If .File.Exists Then
.LoadSettings()
Else
.CookiesDomain = "www.xvideos.com"
.SaveSettings()
End If
End With
OS64 = Environment.Is64BitOperatingSystem
FfmpegFile = "ffmpeg.exe"
FfmpegExists = FfmpegFile.Exists
Domains = New List(Of String)
SiteDomains = New PropertyValue(DomainsDefault, GetType(String), Sub(s) UpdateDomains())
DownloadUHD = New PropertyValue(False)
End Sub
Public Function GetInstance(ByVal What As ISiteSettings.Download) As IPluginContentProvider Implements ISiteSettings.GetInstance
Return New UserData
End Function
Public Sub BeginInit() Implements ISiteSettings.BeginInit
End Sub
Public Sub EndInit() Implements ISiteSettings.EndInit
_Initialized = True
UpdateDomains()
End Sub
Public Sub Load(ByVal XMLValues As IEnumerable(Of KeyValuePair(Of String, String))) Implements ISiteSettings.Load
End Sub
Private _DomainsUpdateInProgress As Boolean = False
Friend Sub UpdateDomains()
If Not _Initialized Then Exit Sub
If Not _DomainsUpdateInProgress Then
_DomainsUpdateInProgress = True
If Not ACheck(SiteDomains.Value) Then SiteDomains.Value = DomainsDefault
Domains.ListAddList(CStr(SiteDomains.Value).Split("|"), LAP.NotContainsOnly, LAP.ClearBeforeAdd)
Domains.ListAddList(DomainsDefault.Split("|"), LAP.NotContainsOnly)
SiteDomains.Value = Domains.ListToString(, "|")
_DomainsUpdateInProgress = False
End If
End Sub
#Region "Downloading"
Public Function Available(ByVal What As ISiteSettings.Download) As Boolean Implements ISiteSettings.Available
Return UseM3U8
End Function
Public Function ReadyToDownload(ByVal What As ISiteSettings.Download) As Boolean Implements ISiteSettings.ReadyToDownload
Return UseM3U8
End Function
Public Sub DownloadStarted(ByVal What As ISiteSettings.Download) Implements ISiteSettings.DownloadStarted
End Sub
Public Sub BeforeStartDownload(ByVal User As Object, ByVal What As ISiteSettings.Download) Implements ISiteSettings.BeforeStartDownload
End Sub
Public Sub AfterDownload(ByVal User As Object, ByVal What As ISiteSettings.Download) Implements ISiteSettings.AfterDownload
End Sub
Public Sub DownloadDone(ByVal What As ISiteSettings.Download) Implements ISiteSettings.DownloadDone
End Sub
#End Region
Public Sub BeginUpdate() Implements ISiteSettings.BeginUpdate
End Sub
Public Sub EndUpdate() Implements ISiteSettings.EndUpdate
End Sub
Public Sub Update() Implements ISiteSettings.Update
UpdateDomains()
Responser.SaveSettings()
End Sub
Public Sub Reset() Implements ISiteSettings.Reset
End Sub
Public Sub OpenSettingsForm() Implements ISiteSettings.OpenSettingsForm
Using f As New SettingsForm(Me)
Design = New XmlFile("Settings\Design_XVIDEOS.xml")
f.ShowDialog()
Design.Dispose()
End Using
End Sub
Public Sub UserOptions(ByRef Options As Object, ByVal OpenForm As Boolean) Implements ISiteSettings.UserOptions
Options = Nothing
End Sub
Public Function GetUserUrl(ByVal UserName As String, ByVal Channel As Boolean) As String Implements ISiteSettings.GetUserUrl
Dim user$ = UserName.Split("_").FirstOrDefault
user &= $"/{UserName.Replace($"{user}_", String.Empty)}"
Return user
End Function
Private Const UserRegexDefault As String = "/(profiles|[\w]*?[-]{0,1}channels)/([^/]+)(\Z|.*?)"
Private Const URD As String = ".*?{0}{1}"
Public Function IsMyUser(ByVal UserURL As String) As ExchangeOptions Implements ISiteSettings.IsMyUser
If Not UserURL.IsEmptyString Then
If Domains.Count > 0 Then
Dim uName$, uOpt$, fStr$
For i% = 0 To Domains.Count - 1
fStr = String.Format(URD, Domains(i), UserRegexDefault)
uName = RegexReplace(UserURL, RParams.DMS(fStr, 2))
If Not uName.IsEmptyString Then
uOpt = RegexReplace(UserURL, RParams.DMS(fStr, 1))
If Not uOpt.IsEmptyString Then Return New ExchangeOptions(Site, $"{uOpt}_{uName}")
End If
Next
End If
End If
Return Nothing
End Function
Public Function IsMyImageVideo(ByVal URL As String) As ExchangeOptions Implements ISiteSettings.IsMyImageVideo
If Not URL.IsEmptyString And Domains.Count > 0 Then
If Domains.Exists(Function(d) URL.Contains(d)) Then Return New ExchangeOptions With {.UserName = URL, .Exists = True}
End If
Return Nothing
End Function
Private Class TempThrower : Implements IThrower
Private Sub ThrowAny() Implements IThrower.ThrowAny
End Sub
End Class
Public Function GetSpecialData(ByVal URL As String, ByVal Path As String, ByVal AskForPath As Boolean) As IEnumerable(Of PluginUserMedia) Implements ISiteSettings.GetSpecialData
If Not URL.IsEmptyString And UseM3U8 Then
Dim f As SFile = Path.CSFileP
f.Name = "video"
f.Extension = "mp4"
#Disable Warning BC40000
If AskForPath Then f = SFile.SaveAs(f,, True, "mp4")
#Enable Warning
If Not f.IsEmptyString Then
Using user As New UserData With {
.LogProvider = Logger,
.Thrower = New TempThrower,
.ESettings = Me,
.DataPath = f.Path
}
With user
.TempMediaList = New List(Of PluginUserMedia) From {UserData.GetVideoData(URL, Responser.Copy, DownloadUHD.Value, String.Empty, Logger)}
If Not .TempMediaList(0).URL.IsEmptyString Then
.Download()
If .TempMediaList(0).DownloadState = PluginUserMedia.States.Downloaded Then
Dim p As PluginUserMedia = .TempMediaList(0)
p.SpecialFolder = p.File
Return {p}
End If
End If
End With
End Using
End If
End If
Return Nothing
End Function
End Class

View File

@@ -0,0 +1,240 @@
' Copyright (C) 2022 Andy
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
' (at your option) any later version.
'
' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY
Imports PersonalUtilities.Functions.RegularExpressions
Imports PersonalUtilities.Functions.XML
Imports PersonalUtilities.Tools.WEB
Imports PersonalUtilities.Tools.WebDocuments.JSON
Imports UStates = SCrawler.Plugin.PluginUserMedia.States
Imports UTypes = SCrawler.Plugin.PluginUserMedia.Types
Public Class UserData : Implements IPluginContentProvider
#Region "Interface declarations"
Public Event ProgressChanged(Count As Integer) Implements IPluginContentProvider.ProgressChanged
Public Event TotalCountChanged(Count As Integer) Implements IPluginContentProvider.TotalCountChanged
Public Property Thrower As IThrower Implements IPluginContentProvider.Thrower
Public Property LogProvider As ILogProvider Implements IPluginContentProvider.LogProvider
Public Property ESettings As ISiteSettings Implements IPluginContentProvider.Settings
Private ReadOnly Property Settings As SiteSettings
Get
Return DirectCast(ESettings, SiteSettings)
End Get
End Property
Public Property Name As String Implements IPluginContentProvider.Name
Public Property ID As String Implements IPluginContentProvider.ID
Public Property ParseUserMediaOnly As Boolean Implements IPluginContentProvider.ParseUserMediaOnly
Public Property UserDescription As String Implements IPluginContentProvider.UserDescription
Public Property ExistingContentList As List(Of PluginUserMedia) Implements IPluginContentProvider.ExistingContentList
Public Property TempPostsList As List(Of String) Implements IPluginContentProvider.TempPostsList
Public Property TempMediaList As List(Of PluginUserMedia) Implements IPluginContentProvider.TempMediaList
Public Property UserExists As Boolean Implements IPluginContentProvider.UserExists
Public Property UserSuspended As Boolean Implements IPluginContentProvider.UserSuspended
Public Property IsSavedPosts As Boolean Implements IPluginContentProvider.IsSavedPosts
Public Property SeparateVideoFolder As Boolean Implements IPluginContentProvider.SeparateVideoFolder
Public Property DataPath As String Implements IPluginContentProvider.DataPath
Public Property PostsNumberLimit As Integer? Implements IPluginContentProvider.PostsNumberLimit
Public Property PostsDateLimit As Date? Implements IPluginContentProvider.PostsDateLimit
#End Region
#Region "Interface exchange options"
Public Sub ExchangeOptionsSet(ByVal Obj As Object) Implements IPluginContentProvider.ExchangeOptionsSet
End Sub
Public Function ExchangeOptionsGet() As Object Implements IPluginContentProvider.ExchangeOptionsGet
Return Nothing
End Function
#End Region
#Region "Interface XML"
Public Sub XmlFieldsSet(ByVal Fields As List(Of KeyValuePair(Of String, String))) Implements IPluginContentProvider.XmlFieldsSet
End Sub
Public Function XmlFieldsGet() As List(Of KeyValuePair(Of String, String)) Implements IPluginContentProvider.XmlFieldsGet
Return Nothing
End Function
#End Region
Private Property Responser As Response
Public Sub GetMedia() Implements IPluginContentProvider.GetMedia
Try
If Not Settings.UseM3U8 Then LogProvider.Add("File [ffmpeg.exe] not found") : Exit Sub
If Not Responser Is Nothing Then Responser.Dispose()
Responser = New Response
Responser.Copy(Settings.Responser)
Dim NextPage% = 0
Dim r$
Dim j As EContainer, jj As EContainer
Dim e As ErrorsDescriber = EDP.ThrowException
Dim user$ = Settings.GetUserUrl(Name, False)
Dim p As PluginUserMedia
Dim EnvirSet As Boolean = False
Do
Thrower.ThrowAny()
r = Responser.GetResponse($"https://www.xvideos.com/{user}/videos/new/{If(NextPage = 0, String.Empty, NextPage)}",, e)
If Not r.IsEmptyString Then
If Not EnvirSet Then UserExists = True : UserSuspended = False : EnvirSet = True
j = JsonDocument.Parse(r).XmlIfNothing
With j
If .Contains("videos") Then
With .Item("videos")
If .Count > 0 Then
NextPage += 1
For Each jj In .Self
p = New PluginUserMedia With {
.PostID = jj.Value("id"),
.URL = $"https://www.xvideos.com{jj.Value("u")}"
}
If Not p.PostID.IsEmptyString And Not jj.Value("u").IsEmptyString Then
If Not TempPostsList.Contains(p.PostID) Then TempPostsList.Add(p.PostID) : TempMediaList.Add(p) Else Exit Do
End If
Next
End If
End With
Else
.Dispose()
Exit Do
End If
.Dispose()
End With
Else
Exit Do
End If
Loop
If TempMediaList.Count > 0 Then
For i% = 0 To TempMediaList.Count - 1
Thrower.ThrowAny()
With TempMediaList(i)
TempMediaList(i) = GetVideoData(.URL, Responser, Settings.DownloadUHD.Value, .PostID, LogProvider)
End With
Next
TempMediaList.RemoveAll(Function(m) m.URL.IsEmptyString)
End If
Catch oex As OperationCanceledException
Catch dex As ObjectDisposedException
Catch ex As Exception
If Responser.StatusCode = Net.HttpStatusCode.NotFound Then
UserExists = False
Else
LogProvider.Add(ex, "[XVIDEOS.UserData.GetMedia]")
End If
Finally
If TempMediaList.ListExists Then TempMediaList.RemoveAll(Function(m) m.URL.IsEmptyString)
End Try
End Sub
Private Structure VSize : Implements IRegExCreator, IComparable(Of VSize)
Friend Size As Integer
Friend Value As String
Private Function CreateFromArray(ByVal ParamsArray() As String) As Object Implements IRegExCreator.CreateFromArray
If ParamsArray.ListExists(2) Then
Size = AConvert(Of Integer)(ParamsArray(0), 0)
Value = ParamsArray(1)
End If
Return Me
End Function
Private Function CompareTo(ByVal Other As VSize) As Integer Implements IComparable(Of VSize).CompareTo
Return Size.CompareTo(Other.Size) * -1
End Function
End Structure
Friend Shared Function GetVideoData(ByVal URL As String, ByVal resp As Response, ByVal DownloadUHD As Boolean,
ByVal ID As String, ByRef Logger As ILogProvider) As PluginUserMedia
Try
If Not URL.IsEmptyString Then
Dim r$ = resp.GetResponse(URL,, EDP.ThrowException)
If Not r.IsEmptyString Then
Dim m$ = RegexReplace(r, M3U8Regex)
If Not m.IsEmptyString Then
Dim appender$ = RegexReplace(m, M3U8Appender)
Dim t$ = RegexReplace(r, VideoTitleRegex)
r = resp.GetResponse(m,, EDP.ThrowException)
If Not r.IsEmptyString Then
Dim ls As List(Of VSize) = FNF.RegexFields(Of VSize)(r, {M3U8Reparse}, {1, 2})
If ls.ListExists And Not DownloadUHD Then ls.RemoveAll(Function(v) v.Size > 1080)
If ls.ListExists Then
ls.Sort()
m = $"{appender}/{ls(0).Value}"
ls.Clear()
Dim pID$ = ID
If pID.IsEmptyString Then pID = RegexReplace(r, VideoID)
If pID.IsEmptyString Then pID = "0"
If Not t.IsEmptyString Then t = t.StringRemoveWinForbiddenSymbols(" ")
If t.IsEmptyString Then
t = pID
Else
If t.Length > 100 Then t = Left(t, 100)
End If
If Not m.IsEmptyString Then
Return New PluginUserMedia With {
.ContentType = UTypes.m3u8,
.PostID = pID,
.URL = m,
.File = $"{t}.mp4",
.SpecialFolder = appender
}
End If
End If
End If
End If
End If
End If
Return Nothing
Catch ex As Exception
Logger.Add(ex, $"[XVIDEOS.UserData.GetVideoData({URL})]")
Return Nothing
End Try
End Function
Public Sub Download() Implements IPluginContentProvider.Download
Try
If TempMediaList.Count > 0 Then
RaiseEvent TotalCountChanged(TempMediaList.Count - 1)
Dim m As PluginUserMedia
Dim f As SFile
Dim DefPath As String = DataPath.CSFilePSN
For i% = 0 To TempMediaList.Count - 1
Thrower.ThrowAny()
m = TempMediaList(i)
f = m.File
f.Path = DefPath
m.DownloadState = UStates.Tried
Try
f = M3U8.Download(m.URL, m.SpecialFolder, Settings.FfmpegFile, f, LogProvider)
m.File = f
m.DownloadState = UStates.Downloaded
Catch ex As Exception
m.DownloadState = UStates.Skipped
End Try
TempMediaList(i) = m
RaiseEvent ProgressChanged(1)
Next
End If
Catch oex As OperationCanceledException
Catch dex As ObjectDisposedException
Catch ex As Exception
LogProvider.Add(ex, $"[XVIDEOS.UserData.Download]")
End Try
End Sub
#Region "IDisposable Support"
Private disposedValue As Boolean = False
Protected Overridable Overloads Sub Dispose(ByVal disposing As Boolean)
If Not disposedValue Then
If disposing Then
If Not Responser Is Nothing Then Responser.Dispose()
If TempMediaList.ListExists Then TempMediaList.Clear()
If TempPostsList.ListExists Then TempPostsList.Clear()
If ExistingContentList.ListExists Then ExistingContentList.Clear()
End If
disposedValue = True
End If
End Sub
Protected Overrides Sub Finalize()
Dispose(False)
MyBase.Finalize()
End Sub
Public Overloads Sub Dispose() Implements IDisposable.Dispose
Dispose(True)
GC.SuppressFinalize(Me)
End Sub
#End Region
End Class

View File

@@ -0,0 +1,3 @@
[*.vb]
# Modifier preferences
file_header_template = Copyright (C) 2022 Andy\nThis program is free software: you can redistribute it and/or modify\nit under the terms of the GNU General Public License as published by\nthe Free Software Foundation, either version 3 of the License, or\n(at your option) any later version.\n\nThis program is distributed in the hope that it will be useful,\nbut WITHOUT ANY WARRANTY; without even the implied warranty of\nMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the\nGNU General Public License for more details.\n\nYou should have received a copy of the GNU General Public License\nalong with this program. If not, see <https://www.gnu.org/licenses/>

View File

@@ -0,0 +1,153 @@
' Copyright (C) 2022 Andy
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
' (at your option) any later version.
'
' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY
Imports System.Runtime.CompilerServices
Namespace Plugin.Attributes
''' <summary>Create a control for a property</summary>
<AttributeUsage(AttributeTargets.Property, AllowMultiple:=False, Inherited:=False)> Public NotInheritable Class PropertyOption : Inherits Attribute
''' <summary>Property name</summary>
Public ReadOnly Property Name As String
''' <summary>Property value type</summary>
Public Property [Type] As Type
Private _ControlText As String
''' <summary>This text will be displayed on the control information.<br/>Default: equals the name (property name)</summary>
Public Property ControlText As String
Get
Return If(String.IsNullOrEmpty(_ControlText), Name, _ControlText)
End Get
Set(ByVal NewText As String)
_ControlText = NewText
End Set
End Property
''' <summary>This tooltip will be displayed on the control.<br/>Default: <see langword="String.Empty"/></summary>
Public Property ControlToolTip As String
''' <summary>CheckBox ThreeStates mode</summary>
Public Property ThreeStates As Boolean = False
''' <summary>Property allows null values</summary>
Public Property AllowNull As Boolean = True
''' <summary>Offset the control from the left border of the form.<br/>Default: 100</summary>
Public Property LeftOffset As Integer = 100
''' <summary>This is an authorization property</summary>
Public Property IsAuth As Boolean = False
''' <summary>Initialize a new property option attribute</summary>
''' <param name="PropertyName">Property name</param>
Public Sub New(<CallerMemberName()> Optional ByVal PropertyName As String = Nothing)
Name = PropertyName
End Sub
End Class
''' <summary>Store property value in settings XML file</summary>
<AttributeUsage(AttributeTargets.Property, AllowMultiple:=False, Inherited:=False)> Public NotInheritable Class PXML : Inherits Attribute
Public ReadOnly ElementName As String
''' <summary>Initialize a new XML attribute</summary>
''' <param name="XMLElementName">XML element name</param>
Public Sub New(<CallerMemberName()> Optional ByVal XMLElementName As String = Nothing)
ElementName = XMLElementName
End Sub
End Class
''' <summary>Special property updater</summary>
<AttributeUsage(AttributeTargets.Method, AllowMultiple:=True, Inherited:=False)> Public NotInheritable Class PropertyUpdater : Inherits Attribute
Public ReadOnly Name As String
Public ReadOnly Dependencies As String()
''' <inheritdoc cref="PropertyUpdater.New(String, String())"/>
Public Sub New(ByVal UpdatingPropertyName As String)
Name = UpdatingPropertyName
End Sub
''' <summary>Initialize a new PropertyUpdater attribute</summary>
''' <param name="UpdatingPropertyName">The name of the property to be updated</param>
Public Sub New(ByVal UpdatingPropertyName As String, ByVal Dependent As String())
Name = UpdatingPropertyName
Dependencies = Dependent
End Sub
End Class
''' <summary>Plugin key</summary>
<AttributeUsage(AttributeTargets.Class, AllowMultiple:=False, Inherited:=False)> Public NotInheritable Class Manifest : Inherits Attribute
Public ReadOnly GUID As String
''' <summary>Initialize a new Manifest attribute</summary>
''' <param name="ClassGuid">Plugin key</param>
Public Sub New(ByVal ClassGuid As String)
GUID = ClassGuid
End Sub
End Class
''' <summary>Special form attribute for settings forms and user creator form</summary>
<AttributeUsage(AttributeTargets.Class, AllowMultiple:=True, Inherited:=False)> Public NotInheritable Class SpecialForm : Inherits Attribute
Public ReadOnly SettingsForm As Boolean
''' <summary>Initialize a new SpecialForm attribute</summary>
''' <param name="IsSettingsForm">
''' <see langword="True"/> - for setting form<br/>
''' <see langword="False"/> - for user creator form
''' </param>
Public Sub New(ByVal IsSettingsForm As Boolean)
SettingsForm = IsSettingsForm
End Sub
End Class
''' <summary>Property provider</summary>
<AttributeUsage(AttributeTargets.Property, AllowMultiple:=True, Inherited:=False)> Public NotInheritable Class Provider : Inherits Attribute
Public ReadOnly Name As String
''' <summary>
''' <see langword="True"/> - form field validation provider. Must return null if the value is invalid.<br/>
''' <see langword="False"/> - only for conversion
''' </summary>
Public FieldsChecker As Boolean = False
''' <summary>Initialize a new Provider attribute. <see cref="IFormatProvider"/> is only allowed</summary>
''' <param name="PropertyName">The name of the property for which this provider is used</param>
Public Sub New(ByVal PropertyName As String)
Name = PropertyName
End Sub
End Class
''' <summary>Sort attribute for settings form</summary>
<AttributeUsage(AttributeTargets.Property, AllowMultiple:=False, Inherited:=False)> Public NotInheritable Class ControlNumber : Inherits Attribute
Public ReadOnly PropertyNumber As String
''' <summary>Initialize a new sort attribute instance for the settings form</summary>
''' <param name="Number">Object position number in the settings form</param>
Public Sub New(ByVal Number As Integer)
PropertyNumber = Number
End Sub
End Class
''' <summary>Attribute for properties values validation methods</summary>
<AttributeUsage(AttributeTargets.Method, AllowMultiple:=True, Inherited:=False)> Public NotInheritable Class PropertiesDataChecker : Inherits Attribute
Public ReadOnly ComparableNames As String()
''' <summary>Initialize a new PropertiesDataChecker attribute.</summary>
''' <param name="Names">Array of the property names</param>
Public Sub New(ByVal Names As String())
ComparableNames = Names
End Sub
End Class
''' <summary>This attribute specifies that users should be downloaded on a separate thread.</summary>
<AttributeUsage(AttributeTargets.Class, AllowMultiple:=False, Inherited:=False)> Public NotInheritable Class SeparatedTasks : Inherits Attribute
Public ReadOnly TasksCount As Integer
''' <summary>Initialize a new SeparatedTasks attribute.</summary>
''' <param name="JobsCount">
''' Predefined task counter.<br/>
''' <see cref="TaskCounter"/> will take precedence if it is defined.
''' </param>
Public Sub New(Optional ByVal JobsCount As Integer = -1)
TasksCount = JobsCount
End Sub
End Class
''' <summary>A property attribute that specifies how many users should be downloaded at the same time in one thread</summary>
<AttributeUsage(AttributeTargets.Property, AllowMultiple:=False, Inherited:=False)> Public NotInheritable Class TaskCounter : Inherits Attribute
End Class
''' <summary>This attribute indicates that the plugin has a SavedPosts environment</summary>
<AttributeUsage(AttributeTargets.Class, AllowMultiple:=False, Inherited:=False)> Public NotInheritable Class SavedPosts : Inherits Attribute
End Class
''' <summary>This is an attribute of the UserData instance. Specifies that the default internal SCrawler downloader should be used.</summary>
<AttributeUsage(AttributeTargets.Class, AllowMultiple:=False, Inherited:=False)> Public NotInheritable Class UseInternalDownloader : Inherits Attribute
End Class
''' <summary>GitHub plugin info</summary>
<AttributeUsage(AttributeTargets.Assembly, AllowMultiple:=False, Inherited:=False)> Public NotInheritable Class Github : Inherits Attribute
Public ReadOnly UserName As String
Public ReadOnly Repository As String
''' <summary>Initialize a new Github attribute.</summary>
''' <param name="Name">Developer GitHub username</param>
''' <param name="RepoName">Plugin repository name</param>
Public Sub New(ByVal Name As String, ByVal RepoName As String)
UserName = Name
Repository = RepoName
End Sub
End Class
End Namespace

View File

@@ -0,0 +1,37 @@
' Copyright (C) 2022 Andy
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
' (at your option) any later version.
'
' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY
Namespace Plugin
Public Interface IPluginContentProvider : Inherits IDisposable
Event ProgressChanged(ByVal Count As Integer)
Event TotalCountChanged(ByVal Count As Integer)
Property Thrower As IThrower
Property LogProvider As ILogProvider
Property Settings As ISiteSettings
Property Name As String
Property ID As String
Property ParseUserMediaOnly As Boolean
Property UserDescription As String
Property ExistingContentList As List(Of PluginUserMedia)
Property TempPostsList As List(Of String)
Property TempMediaList As List(Of PluginUserMedia)
Property UserExists As Boolean
Property UserSuspended As Boolean
Property IsSavedPosts As Boolean
Property SeparateVideoFolder As Boolean
Property DataPath As String
Property PostsNumberLimit As Integer?
Property PostsDateLimit As Date?
Function ExchangeOptionsGet() As Object
Sub ExchangeOptionsSet(ByVal Obj As Object)
Sub XmlFieldsSet(ByVal Fields As List(Of KeyValuePair(Of String, String)))
Function XmlFieldsGet() As List(Of KeyValuePair(Of String, String))
Sub GetMedia()
Sub Download()
End Interface
End Namespace

View File

@@ -0,0 +1,50 @@
' Copyright (C) 2022 Andy
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
' (at your option) any later version.
'
' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY
Imports System.Drawing
Namespace Plugin
Public Interface ISiteSettings
Enum Download As Integer
Main = 0
SavedPosts = 1
Channel = 2
End Enum
ReadOnly Property Icon As Icon
ReadOnly Property Image As Image
ReadOnly Property Site As String
Property Logger As ILogProvider
Function GetUserUrl(ByVal UserName As String, ByVal Channel As Boolean) As String
Function IsMyUser(ByVal UserURL As String) As ExchangeOptions
Function IsMyImageVideo(ByVal URL As String) As ExchangeOptions
Function GetSpecialData(ByVal URL As String, ByVal Path As String, ByVal AskForPath As Boolean) As IEnumerable(Of PluginUserMedia)
Function GetInstance(ByVal What As Download) As IPluginContentProvider
#Region "XML Support"
Sub Load(ByVal XMLValues As IEnumerable(Of KeyValuePair(Of String, String)))
#End Region
#Region "Initialization"
Sub BeginInit()
Sub EndInit()
Sub BeginUpdate()
Sub EndUpdate()
#End Region
#Region "Site availability"
Function Available(ByVal What As Download) As Boolean
Function ReadyToDownload(ByVal What As Download) As Boolean
#End Region
#Region "Downloading"
Sub DownloadStarted(ByVal What As Download)
Sub BeforeStartDownload(ByVal User As Object, ByVal What As Download)
Sub AfterDownload(ByVal User As Object, ByVal What As Download)
Sub DownloadDone(ByVal What As Download)
#End Region
Sub Update()
Sub Reset()
Sub OpenSettingsForm()
Sub UserOptions(ByRef Options As Object, ByVal OpenForm As Boolean)
End Interface
End Namespace

View File

@@ -0,0 +1,13 @@
'------------------------------------------------------------------------------
' <auto-generated>
' This code was generated by a tool.
' Runtime Version:4.0.30319.42000
'
' Changes to this file may cause incorrect behavior and will be lost if
' the code is regenerated.
' </auto-generated>
'------------------------------------------------------------------------------
Option Strict On
Option Explicit On

View File

@@ -0,0 +1,10 @@
<?xml version="1.0" encoding="utf-8"?>
<MyApplicationData xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<MySubMain>false</MySubMain>
<SingleInstance>false</SingleInstance>
<ShutdownMode>0</ShutdownMode>
<EnableVisualStyles>true</EnableVisualStyles>
<AuthenticationMode>0</AuthenticationMode>
<ApplicationType>1</ApplicationType>
<SaveMySettingsOnExit>true</SaveMySettingsOnExit>
</MyApplicationData>

View File

@@ -0,0 +1,37 @@
Imports System.Resources
Imports System
Imports System.Reflection
Imports System.Runtime.InteropServices
' General Information about an assembly is controlled through the following
' set of attributes. Change these attribute values to modify the information
' associated with an assembly.
' Review the values of the assembly attributes
<Assembly: AssemblyTitle("SCrawler plugin provider")>
<Assembly: AssemblyDescription("Plugin provider for SCrawler")>
<Assembly: AssemblyCompany("AndyProgram")>
<Assembly: AssemblyProduct("SCrawler.PluginProvider")>
<Assembly: AssemblyCopyright("Copyright © 2022")>
<Assembly: AssemblyTrademark("AndyProgram")>
<Assembly: ComVisible(False)>
'The following GUID is for the ID of the typelib if this project is exposed to COM
<Assembly: Guid("1f6c9516-360e-4a2b-80d4-ae0d9a4e5cfd")>
' Version information for an assembly consists of the following four values:
'
' Major Version
' Minor Version
' Build Number
' Revision
'
' You can specify all the values or you can default the Build and Revision Numbers
' by using the '*' as shown below:
' <Assembly: AssemblyVersion("1.0.*")>
<Assembly: AssemblyVersion("3.0.0.2")>
<Assembly: AssemblyFileVersion("3.0.0.2")>
<Assembly: NeutralResourcesLanguage("en")>

View File

@@ -0,0 +1,63 @@
'------------------------------------------------------------------------------
' <auto-generated>
' This code was generated by a tool.
' Runtime Version:4.0.30319.42000
'
' Changes to this file may cause incorrect behavior and will be lost if
' the code is regenerated.
' </auto-generated>
'------------------------------------------------------------------------------
Option Strict On
Option Explicit On
Imports System
Namespace My.Resources
'This class was auto-generated by the StronglyTypedResourceBuilder
'class via a tool like ResGen or Visual Studio.
'To add or remove a member, edit your .ResX file then rerun ResGen
'with the /str option, or rebuild your VS project.
'''<summary>
''' A strongly-typed resource class, for looking up localized strings, etc.
'''</summary>
<Global.System.CodeDom.Compiler.GeneratedCodeAttribute("System.Resources.Tools.StronglyTypedResourceBuilder", "16.0.0.0"), _
Global.System.Diagnostics.DebuggerNonUserCodeAttribute(), _
Global.System.Runtime.CompilerServices.CompilerGeneratedAttribute(), _
Global.Microsoft.VisualBasic.HideModuleNameAttribute()> _
Friend Module Resources
Private resourceMan As Global.System.Resources.ResourceManager
Private resourceCulture As Global.System.Globalization.CultureInfo
'''<summary>
''' Returns the cached ResourceManager instance used by this class.
'''</summary>
<Global.System.ComponentModel.EditorBrowsableAttribute(Global.System.ComponentModel.EditorBrowsableState.Advanced)> _
Friend ReadOnly Property ResourceManager() As Global.System.Resources.ResourceManager
Get
If Object.ReferenceEquals(resourceMan, Nothing) Then
Dim temp As Global.System.Resources.ResourceManager = New Global.System.Resources.ResourceManager("SCrawler.Resources", GetType(Resources).Assembly)
resourceMan = temp
End If
Return resourceMan
End Get
End Property
'''<summary>
''' Overrides the current thread's CurrentUICulture property for all
''' resource lookups using this strongly typed resource class.
'''</summary>
<Global.System.ComponentModel.EditorBrowsableAttribute(Global.System.ComponentModel.EditorBrowsableState.Advanced)> _
Friend Property Culture() As Global.System.Globalization.CultureInfo
Get
Return resourceCulture
End Get
Set
resourceCulture = value
End Set
End Property
End Module
End Namespace

View File

@@ -0,0 +1,117 @@
<?xml version="1.0" encoding="utf-8"?>
<root>
<!--
Microsoft ResX Schema
Version 2.0
The primary goals of this format is to allow a simple XML format
that is mostly human readable. The generation and parsing of the
various data types are done through the TypeConverter classes
associated with the data types.
Example:
... ado.net/XML headers & schema ...
<resheader name="resmimetype">text/microsoft-resx</resheader>
<resheader name="version">2.0</resheader>
<resheader name="reader">System.Resources.ResXResourceReader, System.Windows.Forms, ...</resheader>
<resheader name="writer">System.Resources.ResXResourceWriter, System.Windows.Forms, ...</resheader>
<data name="Name1"><value>this is my long string</value><comment>this is a comment</comment></data>
<data name="Color1" type="System.Drawing.Color, System.Drawing">Blue</data>
<data name="Bitmap1" mimetype="application/x-microsoft.net.object.binary.base64">
<value>[base64 mime encoded serialized .NET Framework object]</value>
</data>
<data name="Icon1" type="System.Drawing.Icon, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
<value>[base64 mime encoded string representing a byte array form of the .NET Framework object]</value>
<comment>This is a comment</comment>
</data>
There are any number of "resheader" rows that contain simple
name/value pairs.
Each data row contains a name, and value. The row also contains a
type or mimetype. Type corresponds to a .NET class that support
text/value conversion through the TypeConverter architecture.
Classes that don't support this are serialized and stored with the
mimetype set.
The mimetype is used for serialized objects, and tells the
ResXResourceReader how to depersist the object. This is currently not
extensible. For a given mimetype the value must be set accordingly:
Note - application/x-microsoft.net.object.binary.base64 is the format
that the ResXResourceWriter will generate, however the reader can
read any of the formats listed below.
mimetype: application/x-microsoft.net.object.binary.base64
value : The object must be serialized with
: System.Serialization.Formatters.Binary.BinaryFormatter
: and then encoded with base64 encoding.
mimetype: application/x-microsoft.net.object.soap.base64
value : The object must be serialized with
: System.Runtime.Serialization.Formatters.Soap.SoapFormatter
: and then encoded with base64 encoding.
mimetype: application/x-microsoft.net.object.bytearray.base64
value : The object must be serialized into a byte array
: using a System.ComponentModel.TypeConverter
: and then encoded with base64 encoding.
-->
<xsd:schema id="root" xmlns="" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:msdata="urn:schemas-microsoft-com:xml-msdata">
<xsd:element name="root" msdata:IsDataSet="true">
<xsd:complexType>
<xsd:choice maxOccurs="unbounded">
<xsd:element name="metadata">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="value" type="xsd:string" minOccurs="0" />
</xsd:sequence>
<xsd:attribute name="name" type="xsd:string" />
<xsd:attribute name="type" type="xsd:string" />
<xsd:attribute name="mimetype" type="xsd:string" />
</xsd:complexType>
</xsd:element>
<xsd:element name="assembly">
<xsd:complexType>
<xsd:attribute name="alias" type="xsd:string" />
<xsd:attribute name="name" type="xsd:string" />
</xsd:complexType>
</xsd:element>
<xsd:element name="data">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="value" type="xsd:string" minOccurs="0" msdata:Ordinal="1" />
<xsd:element name="comment" type="xsd:string" minOccurs="0" msdata:Ordinal="2" />
</xsd:sequence>
<xsd:attribute name="name" type="xsd:string" msdata:Ordinal="1" />
<xsd:attribute name="type" type="xsd:string" msdata:Ordinal="3" />
<xsd:attribute name="mimetype" type="xsd:string" msdata:Ordinal="4" />
</xsd:complexType>
</xsd:element>
<xsd:element name="resheader">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="value" type="xsd:string" minOccurs="0" msdata:Ordinal="1" />
</xsd:sequence>
<xsd:attribute name="name" type="xsd:string" use="required" />
</xsd:complexType>
</xsd:element>
</xsd:choice>
</xsd:complexType>
</xsd:element>
</xsd:schema>
<resheader name="resmimetype">
<value>text/microsoft-resx</value>
</resheader>
<resheader name="version">
<value>2.0</value>
</resheader>
<resheader name="reader">
<value>System.Resources.ResXResourceReader, System.Windows.Forms, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
</resheader>
<resheader name="writer">
<value>System.Resources.ResXResourceWriter, System.Windows.Forms, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
</resheader>
</root>

View File

@@ -0,0 +1,73 @@
'------------------------------------------------------------------------------
' <auto-generated>
' This code was generated by a tool.
' Runtime Version:4.0.30319.42000
'
' Changes to this file may cause incorrect behavior and will be lost if
' the code is regenerated.
' </auto-generated>
'------------------------------------------------------------------------------
Option Strict On
Option Explicit On
Namespace My
<Global.System.Runtime.CompilerServices.CompilerGeneratedAttribute(), _
Global.System.CodeDom.Compiler.GeneratedCodeAttribute("Microsoft.VisualStudio.Editors.SettingsDesigner.SettingsSingleFileGenerator", "16.10.0.0"), _
Global.System.ComponentModel.EditorBrowsableAttribute(Global.System.ComponentModel.EditorBrowsableState.Advanced)> _
Partial Friend NotInheritable Class MySettings
Inherits Global.System.Configuration.ApplicationSettingsBase
Private Shared defaultInstance As MySettings = CType(Global.System.Configuration.ApplicationSettingsBase.Synchronized(New MySettings()),MySettings)
#Region "My.Settings Auto-Save Functionality"
#If _MyType = "WindowsForms" Then
Private Shared addedHandler As Boolean
Private Shared addedHandlerLockObject As New Object
<Global.System.Diagnostics.DebuggerNonUserCodeAttribute(), Global.System.ComponentModel.EditorBrowsableAttribute(Global.System.ComponentModel.EditorBrowsableState.Advanced)> _
Private Shared Sub AutoSaveSettings(sender As Global.System.Object, e As Global.System.EventArgs)
If My.Application.SaveMySettingsOnExit Then
My.Settings.Save()
End If
End Sub
#End If
#End Region
Public Shared ReadOnly Property [Default]() As MySettings
Get
#If _MyType = "WindowsForms" Then
If Not addedHandler Then
SyncLock addedHandlerLockObject
If Not addedHandler Then
AddHandler My.Application.Shutdown, AddressOf AutoSaveSettings
addedHandler = True
End If
End SyncLock
End If
#End If
Return defaultInstance
End Get
End Property
End Class
End Namespace
Namespace My
<Global.Microsoft.VisualBasic.HideModuleNameAttribute(), _
Global.System.Diagnostics.DebuggerNonUserCodeAttribute(), _
Global.System.Runtime.CompilerServices.CompilerGeneratedAttribute()> _
Friend Module MySettingsProperty
<Global.System.ComponentModel.Design.HelpKeywordAttribute("My.Settings")> _
Friend ReadOnly Property Settings() As Global.SCrawler.My.MySettings
Get
Return Global.SCrawler.My.MySettings.Default
End Get
End Property
End Module
End Namespace

View File

@@ -0,0 +1,7 @@
<?xml version='1.0' encoding='utf-8'?>
<SettingsFile xmlns="http://schemas.microsoft.com/VisualStudio/2004/01/settings" CurrentProfile="(Default)" UseMySettingsClassName="true">
<Profiles>
<Profile Name="(Default)" />
</Profiles>
<Settings />
</SettingsFile>

View File

@@ -0,0 +1,16 @@
' Copyright (C) 2022 Andy
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
' (at your option) any later version.
'
' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY
Namespace Plugin
Public Interface ILogProvider
Overloads Sub Add(ByVal Message As String)
Overloads Sub Add(ByVal ex As Exception, ByVal Message As String,
Optional ByVal ShowMainMsg As Boolean = False, Optional ByVal ShowErrorMsg As Boolean = False,
Optional ByVal SendInLog As Boolean = True)
End Interface
End Namespace

View File

@@ -0,0 +1,13 @@
' Copyright (C) 2022 Andy
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
' (at your option) any later version.
'
' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY
Namespace Plugin
Public Interface IThrower
Sub ThrowAny()
End Interface
End Namespace

View File

@@ -0,0 +1,25 @@
' Copyright (C) 2022 Andy
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
' (at your option) any later version.
'
' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY
Namespace Plugin
Public Structure ExchangeOptions
Public UserName As String
Public SiteName As String
Public HostKey As String
Public IsChannel As Boolean
Public Exists As Boolean
Public Sub New(ByVal Site As String, ByVal _Name As String)
UserName = _Name
SiteName = Site
End Sub
Public Sub New(ByVal Site As String, ByVal _Name As String, ByVal _IsChannel As Boolean)
Me.New(Site, _Name)
IsChannel = _IsChannel
End Sub
End Structure
End Namespace

View File

@@ -0,0 +1,30 @@
' Copyright (C) 2022 Andy
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
' (at your option) any later version.
'
' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY
Namespace Plugin
Public Structure PluginUserMedia
Enum Types As Integer
Undefined = 0
[Picture] = 1
[Video] = 2
[Text] = 3
VideoPre = 10
GIF = 50
m3u8 = 100
End Enum
Enum States As Integer : Unknown = 0 : Tried = 1 : Downloaded = 2 : Skipped = 3 : End Enum
Public ContentType As Integer
Public URL As String
Public MD5 As String
Public File As String
Public DownloadState As Integer
Public PostID As String
Public PostDate As Date?
Public SpecialFolder As String
End Structure
End Namespace

View File

@@ -0,0 +1,18 @@
' Copyright (C) 2022 Andy
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
' (at your option) any later version.
'
' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY
Namespace Plugin
Public Structure PropertyData
Public ReadOnly Name As String
Public ReadOnly Value As Object
Public Sub New(ByVal _Name As String, ByVal _Value As Object)
Name = _Name
Value = _Value
End Sub
End Structure
End Namespace

View File

@@ -0,0 +1,58 @@
' Copyright (C) 2022 Andy
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
' (at your option) any later version.
'
' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY
Namespace Plugin
Public NotInheritable Class PropertyValue : Implements IPropertyValue
Public Event ValueChanged As IPropertyValue.ValueChangedEventHandler Implements IPropertyValue.ValueChanged
Public Property [Type] As Type Implements IPropertyValue.Type
Public Property OnChangeFunction As IPropertyValue.ValueChangedEventHandler
''' <inheritdoc cref="PropertyValue.New(Object, Type, ByRef IPropertyValue.ValueChangedEventHandler)"/>
''' <exception cref="ArgumentNullException"></exception>
Public Sub New(ByVal InitValue As Object)
_Value = InitValue
If IsNothing(InitValue) Then
Throw New ArgumentNullException("InitValue", "InitValue cannot be null")
Else
[Type] = _Value.GetType
End If
End Sub
''' <inheritdoc cref="PropertyValue.New(Object, Type, ByRef IPropertyValue.ValueChangedEventHandler)"/>
Public Sub New(ByVal InitValue As Object, ByVal T As Type)
_Value = InitValue
[Type] = T
End Sub
''' <summary>New property value instance</summary>
''' <param name="InitValue">Initialization value</param>
''' <param name="T">Value type</param>
''' <param name="RFunction">CallBack function on value change</param>
Public Sub New(ByVal InitValue As Object, ByVal T As Type, ByRef RFunction As IPropertyValue.ValueChangedEventHandler)
Me.New(InitValue, T)
OnChangeFunction = RFunction
End Sub
Private _Value As Object
Public Property Value As Object Implements IPropertyValue.Value
Get
Return _Value
End Get
Set(ByVal NewValue As Object)
_Value = NewValue
If Not OnChangeFunction Is Nothing Then OnChangeFunction.Invoke(Value)
RaiseEvent ValueChanged(_Value)
End Set
End Property
End Class
Public Interface IPropertyValue
''' <summary>Event for internal exchange</summary>
''' <param name="Value">New value</param>
Event ValueChanged(ByVal Value As Object)
''' <summary>Value type</summary>
Property [Type] As Type
''' <summary>Property value</summary>
Property Value As Object
End Interface
End Namespace

View File

@@ -0,0 +1,151 @@
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="15.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Import Project="$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props" Condition="Exists('$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props')" />
<PropertyGroup>
<Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
<Platform Condition=" '$(Platform)' == '' ">AnyCPU</Platform>
<ProjectGuid>{D4650F6B-5A54-44B6-999B-6C675B7116B1}</ProjectGuid>
<OutputType>Library</OutputType>
<RootNamespace>SCrawler</RootNamespace>
<AssemblyName>SCrawler.PluginProvider</AssemblyName>
<FileAlignment>512</FileAlignment>
<MyType>Windows</MyType>
<TargetFrameworkVersion>v4.6.1</TargetFrameworkVersion>
<Deterministic>true</Deterministic>
</PropertyGroup>
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
<DebugSymbols>true</DebugSymbols>
<DebugType>full</DebugType>
<DefineDebug>true</DefineDebug>
<DefineTrace>true</DefineTrace>
<OutputPath>bin\Debug\</OutputPath>
<DocumentationFile>
</DocumentationFile>
<NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
</PropertyGroup>
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
<DebugType>pdbonly</DebugType>
<DefineDebug>false</DefineDebug>
<DefineTrace>true</DefineTrace>
<Optimize>true</Optimize>
<OutputPath>bin\Release\</OutputPath>
<DocumentationFile>
</DocumentationFile>
<NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
</PropertyGroup>
<PropertyGroup>
<OptionExplicit>On</OptionExplicit>
</PropertyGroup>
<PropertyGroup>
<OptionCompare>Binary</OptionCompare>
</PropertyGroup>
<PropertyGroup>
<OptionStrict>Off</OptionStrict>
</PropertyGroup>
<PropertyGroup>
<OptionInfer>On</OptionInfer>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Debug|x64'">
<DebugSymbols>true</DebugSymbols>
<DefineDebug>true</DefineDebug>
<DefineTrace>true</DefineTrace>
<OutputPath>bin\x64\Debug\</OutputPath>
<NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
<DebugType>full</DebugType>
<PlatformTarget>x64</PlatformTarget>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Release|x64'">
<DefineTrace>true</DefineTrace>
<OutputPath>bin\x64\Release\</OutputPath>
<Optimize>true</Optimize>
<NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
<DebugType>pdbonly</DebugType>
<PlatformTarget>x64</PlatformTarget>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Debug|x86'">
<DebugSymbols>true</DebugSymbols>
<DefineDebug>true</DefineDebug>
<DefineTrace>true</DefineTrace>
<OutputPath>bin\x86\Debug\</OutputPath>
<NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
<DebugType>full</DebugType>
<PlatformTarget>x86</PlatformTarget>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Release|x86'">
<DefineTrace>true</DefineTrace>
<OutputPath>bin\x86\Release\</OutputPath>
<Optimize>true</Optimize>
<NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
<DebugType>pdbonly</DebugType>
<PlatformTarget>x86</PlatformTarget>
</PropertyGroup>
<ItemGroup>
<Reference Include="System" />
<Reference Include="System.Data" />
<Reference Include="System.Drawing" />
<Reference Include="System.Xml" />
<Reference Include="System.Core" />
<Reference Include="System.Xml.Linq" />
<Reference Include="System.Data.DataSetExtensions" />
<Reference Include="System.Net.Http" />
</ItemGroup>
<ItemGroup>
<Import Include="Microsoft.VisualBasic" />
<Import Include="System" />
<Import Include="System.Collections" />
<Import Include="System.Collections.Generic" />
<Import Include="System.Data" />
<Import Include="System.Diagnostics" />
<Import Include="System.Linq" />
<Import Include="System.Xml.Linq" />
<Import Include="System.Threading.Tasks" />
</ItemGroup>
<ItemGroup>
<Compile Include="Attributes\Attributes.vb" />
<Compile Include="Objects\ExchangeOptions.vb" />
<Compile Include="ObjectInterfaces\ILogProvider.vb" />
<Compile Include="Interfaces\IPluginContentProvider.vb" />
<Compile Include="Objects\PluginUserMedia.vb" />
<Compile Include="Interfaces\ISiteSettings.vb" />
<Compile Include="ObjectInterfaces\IThrower.vb" />
<Compile Include="My Project\AssemblyInfo.vb" />
<Compile Include="My Project\Application.Designer.vb">
<AutoGen>True</AutoGen>
<DependentUpon>Application.myapp</DependentUpon>
<DesignTime>True</DesignTime>
</Compile>
<Compile Include="My Project\Resources.Designer.vb">
<AutoGen>True</AutoGen>
<DesignTime>True</DesignTime>
<DependentUpon>Resources.resx</DependentUpon>
</Compile>
<Compile Include="My Project\Settings.Designer.vb">
<AutoGen>True</AutoGen>
<DependentUpon>Settings.settings</DependentUpon>
<DesignTimeSharedInput>True</DesignTimeSharedInput>
</Compile>
<Compile Include="Objects\PropertyData.vb" />
<Compile Include="Objects\PropertyValue.vb" />
</ItemGroup>
<ItemGroup>
<EmbeddedResource Include="My Project\Resources.resx">
<Generator>VbMyResourcesResXFileCodeGenerator</Generator>
<LastGenOutput>Resources.Designer.vb</LastGenOutput>
<CustomToolNamespace>My.Resources</CustomToolNamespace>
<SubType>Designer</SubType>
</EmbeddedResource>
</ItemGroup>
<ItemGroup>
<None Include=".editorconfig" />
<None Include="My Project\Application.myapp">
<Generator>MyApplicationCodeGenerator</Generator>
<LastGenOutput>Application.Designer.vb</LastGenOutput>
</None>
<None Include="My Project\Settings.settings">
<Generator>SettingsSingleFileGenerator</Generator>
<CustomToolNamespace>My</CustomToolNamespace>
<LastGenOutput>Settings.Designer.vb</LastGenOutput>
</None>
</ItemGroup>
<Import Project="$(MSBuildToolsPath)\Microsoft.VisualBasic.targets" />
</Project>

View File

@@ -11,15 +11,16 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Solution Items", "Solution
ProjectSection(SolutionItems) = preProject ProjectSection(SolutionItems) = preProject
.gitignore = .gitignore .gitignore = .gitignore
Changelog.md = Changelog.md Changelog.md = Changelog.md
Info\InstaAlgo.txt = Info\InstaAlgo.txt
Info\InstagramInfo.txt = Info\InstagramInfo.txt
README.md = README.md README.md = README.md
Info\RedditUrlsInfo.txt = Info\RedditUrlsInfo.txt
ToDo.txt = ToDo.txt ToDo.txt = ToDo.txt
Info\TwitterNewAlgo.txt = Info\TwitterNewAlgo.txt
Info\TwitterUrlsInfo.txt = Info\TwitterUrlsInfo.txt
EndProjectSection EndProjectSection
EndProject EndProject
Project("{F184B08F-C81C-45F6-A57F-5ABD9991F28F}") = "SCrawler.PluginProvider", "SCrawler.PluginProvider\SCrawler.PluginProvider.vbproj", "{D4650F6B-5A54-44B6-999B-6C675B7116B1}"
EndProject
Project("{F184B08F-C81C-45F6-A57F-5ABD9991F28F}") = "SCrawler.Plugin.LPSG", "SCrawler.Plugin.LPSG\SCrawler.Plugin.LPSG.vbproj", "{22A130B2-DDF4-4FB5-BA38-E5DB4CF1B8A2}"
EndProject
Project("{F184B08F-C81C-45F6-A57F-5ABD9991F28F}") = "SCrawler.Plugin.XVIDEOS", "SCrawler.Plugin.XVIDEOS\SCrawler.Plugin.XVIDEOS.vbproj", "{CCCF47F4-C97C-4193-AC4B-C56DF2F9AA8A}"
EndProject
Global Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU Debug|Any CPU = Debug|Any CPU
@@ -54,6 +55,42 @@ Global
{8405896B-2685-4916-BC93-1FB514C323A9}.Release|x64.Build.0 = Release|x64 {8405896B-2685-4916-BC93-1FB514C323A9}.Release|x64.Build.0 = Release|x64
{8405896B-2685-4916-BC93-1FB514C323A9}.Release|x86.ActiveCfg = Release|x86 {8405896B-2685-4916-BC93-1FB514C323A9}.Release|x86.ActiveCfg = Release|x86
{8405896B-2685-4916-BC93-1FB514C323A9}.Release|x86.Build.0 = Release|x86 {8405896B-2685-4916-BC93-1FB514C323A9}.Release|x86.Build.0 = Release|x86
{D4650F6B-5A54-44B6-999B-6C675B7116B1}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{D4650F6B-5A54-44B6-999B-6C675B7116B1}.Debug|Any CPU.Build.0 = Debug|Any CPU
{D4650F6B-5A54-44B6-999B-6C675B7116B1}.Debug|x64.ActiveCfg = Debug|x64
{D4650F6B-5A54-44B6-999B-6C675B7116B1}.Debug|x64.Build.0 = Debug|x64
{D4650F6B-5A54-44B6-999B-6C675B7116B1}.Debug|x86.ActiveCfg = Debug|x86
{D4650F6B-5A54-44B6-999B-6C675B7116B1}.Debug|x86.Build.0 = Debug|x86
{D4650F6B-5A54-44B6-999B-6C675B7116B1}.Release|Any CPU.ActiveCfg = Release|Any CPU
{D4650F6B-5A54-44B6-999B-6C675B7116B1}.Release|Any CPU.Build.0 = Release|Any CPU
{D4650F6B-5A54-44B6-999B-6C675B7116B1}.Release|x64.ActiveCfg = Release|x64
{D4650F6B-5A54-44B6-999B-6C675B7116B1}.Release|x64.Build.0 = Release|x64
{D4650F6B-5A54-44B6-999B-6C675B7116B1}.Release|x86.ActiveCfg = Release|x86
{D4650F6B-5A54-44B6-999B-6C675B7116B1}.Release|x86.Build.0 = Release|x86
{22A130B2-DDF4-4FB5-BA38-E5DB4CF1B8A2}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{22A130B2-DDF4-4FB5-BA38-E5DB4CF1B8A2}.Debug|Any CPU.Build.0 = Debug|Any CPU
{22A130B2-DDF4-4FB5-BA38-E5DB4CF1B8A2}.Debug|x64.ActiveCfg = Debug|x64
{22A130B2-DDF4-4FB5-BA38-E5DB4CF1B8A2}.Debug|x64.Build.0 = Debug|x64
{22A130B2-DDF4-4FB5-BA38-E5DB4CF1B8A2}.Debug|x86.ActiveCfg = Debug|x86
{22A130B2-DDF4-4FB5-BA38-E5DB4CF1B8A2}.Debug|x86.Build.0 = Debug|x86
{22A130B2-DDF4-4FB5-BA38-E5DB4CF1B8A2}.Release|Any CPU.ActiveCfg = Release|Any CPU
{22A130B2-DDF4-4FB5-BA38-E5DB4CF1B8A2}.Release|Any CPU.Build.0 = Release|Any CPU
{22A130B2-DDF4-4FB5-BA38-E5DB4CF1B8A2}.Release|x64.ActiveCfg = Release|x64
{22A130B2-DDF4-4FB5-BA38-E5DB4CF1B8A2}.Release|x64.Build.0 = Release|x64
{22A130B2-DDF4-4FB5-BA38-E5DB4CF1B8A2}.Release|x86.ActiveCfg = Release|x86
{22A130B2-DDF4-4FB5-BA38-E5DB4CF1B8A2}.Release|x86.Build.0 = Release|x86
{CCCF47F4-C97C-4193-AC4B-C56DF2F9AA8A}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{CCCF47F4-C97C-4193-AC4B-C56DF2F9AA8A}.Debug|Any CPU.Build.0 = Debug|Any CPU
{CCCF47F4-C97C-4193-AC4B-C56DF2F9AA8A}.Debug|x64.ActiveCfg = Debug|x64
{CCCF47F4-C97C-4193-AC4B-C56DF2F9AA8A}.Debug|x64.Build.0 = Debug|x64
{CCCF47F4-C97C-4193-AC4B-C56DF2F9AA8A}.Debug|x86.ActiveCfg = Debug|x86
{CCCF47F4-C97C-4193-AC4B-C56DF2F9AA8A}.Debug|x86.Build.0 = Debug|x86
{CCCF47F4-C97C-4193-AC4B-C56DF2F9AA8A}.Release|Any CPU.ActiveCfg = Release|Any CPU
{CCCF47F4-C97C-4193-AC4B-C56DF2F9AA8A}.Release|Any CPU.Build.0 = Release|Any CPU
{CCCF47F4-C97C-4193-AC4B-C56DF2F9AA8A}.Release|x64.ActiveCfg = Release|x64
{CCCF47F4-C97C-4193-AC4B-C56DF2F9AA8A}.Release|x64.Build.0 = Release|x64
{CCCF47F4-C97C-4193-AC4B-C56DF2F9AA8A}.Release|x86.ActiveCfg = Release|x86
{CCCF47F4-C97C-4193-AC4B-C56DF2F9AA8A}.Release|x86.Build.0 = Release|x86
EndGlobalSection EndGlobalSection
GlobalSection(SolutionProperties) = preSolution GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE HideSolutionNode = FALSE

View File

@@ -0,0 +1,67 @@
' Copyright (C) 2022 Andy
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
' (at your option) any later version.
'
' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY
Imports System.Net
Imports PersonalUtilities.Functions.RegularExpressions
Namespace API.Base
Friend NotInheritable Class DownDetector
Private Shared ReadOnly Property Params As New RParams("x:.'([\S]+?)',.y:.(\d+)", -1, Nothing, RegexReturn.List)
Private Sub New()
End Sub
Friend Structure Data : Implements IRegExCreator, IComparable(Of Data)
Friend [Date] As Date
Friend Value As Integer
Friend Function CreateFromArray(ByVal ParamsArray() As String) As Object Implements IRegExCreator.CreateFromArray
If ParamsArray.ListExists Then
Try : [Date] = Date.Parse(ParamsArray(0)) : Catch : End Try
If ParamsArray.Length > 1 Then Value = AConvert(Of Integer)(ParamsArray(1), 0)
End If
Return Me
End Function
Public Overrides Function ToString() As String
Return $"{AConvert(Of String)([Date], ADateTime.Formats.BaseDateTime, String.Empty)} [{Value}]"
End Function
Friend Function CompareTo(ByVal Other As Data) As Integer Implements IComparable(Of Data).CompareTo
Return [Date].CompareTo(Other.Date) * -1
End Function
End Structure
Friend Shared Function GetData(ByVal Site As String) As List(Of Data)
Try
Dim l As List(Of Data) = Nothing
Dim l2 As List(Of Data) = Nothing
Using w As New WebClient
Dim r$ = w.DownloadString($"https://downdetector.co.uk/status/{Site}/")
If Not r.IsEmptyString Then
l = FNF.RegexFields(Of Data)(r, {Params}, {1, 2})
If l.ListExists(2) Then
l.Sort()
l2 = New List(Of Data)
Dim d As Data
Dim eDates As New List(Of Date)
Dim MaxValue As Func(Of Date, Integer) = Function(dd) (From ddd In l Where ddd.Date = dd Select ddd.Value).DefaultIfEmpty(0).Max
For i% = 0 To l.Count - 1
If Not eDates.Contains(l(i).Date) Then
d = l(i)
d.Value = MaxValue(d.Date)
l2.Add(d)
eDates.Add(d.Date)
End If
Next
eDates.Clear()
l.Clear()
l2.Sort()
End If
End If
End Using
Return l2
Catch ex As Exception
Return ErrorsDescriber.Execute(EDP.SendInLog + EDP.ReturnValue, ex, $"[DownDetector.GetData({Site})]")
End Try
End Function
End Class
End Namespace

View File

@@ -0,0 +1,61 @@
' Copyright (C) 2022 Andy
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
' (at your option) any later version.
'
' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY
Imports SCrawler.Plugin.Hosts
Imports System.Threading
Imports PersonalUtilities.Forms.Toolbars
Imports PDownload = SCrawler.Plugin.ISiteSettings.Download
Namespace API.Base
Friend NotInheritable Class ProfileSaved
Private ReadOnly Property HOST As SettingsHost
Private ReadOnly Property Progress As MyProgress
Friend Sub New(ByRef h As SettingsHost, ByRef Bar As MyProgress)
HOST = h
Progress = Bar
End Sub
Friend Sub Download(ByVal Token As CancellationToken)
Try
If HOST.Source.ReadyToDownload(PDownload.SavedPosts) Then
If HOST.Available(PDownload.SavedPosts) Then
HOST.DownloadStarted(PDownload.SavedPosts)
Dim u As New UserInfo With {.Plugin = HOST.Key, .Site = HOST.Name, .SpecialPath = HOST.SavedPostsPath}
Using user As IUserData = HOST.GetInstance(PDownload.SavedPosts, Nothing, False, False)
If Not user Is Nothing AndAlso (Not user.Name.IsEmptyString Or Not HOST.IsMyClass) Then
u.Name = user.Name
With DirectCast(user, UserDataBase).User
u.IsChannel = .IsChannel
u.UpdateUserFile()
End With
With DirectCast(user, UserDataBase)
.User = u
.LoadUserInformation()
.IsSavedPosts = True
.Progress = Progress
If Not .FileExists Then .UpdateUserInformation()
End With
HOST.BeforeStartDownload(user, PDownload.SavedPosts)
user.DownloadData(Token)
Progress.InformationTemporary = $"Images: {user.DownloadedPictures(False)}; Videos: {user.DownloadedVideos(False)}"
HOST.AfterDownload(user, PDownload.SavedPosts)
End If
End Using
Else
Progress.InformationTemporary = $"Host [{HOST.Name}] is unavailable"
End If
Else
Progress.InformationTemporary = $"Host [{HOST.Name}] is nor ready"
End If
Catch ex As Exception
Progress.InformationTemporary = $"{HOST.Name} downloading error"
ErrorsDescriber.Execute(EDP.SendInLog, ex, $"[API.Base.ProfileSaved.Download({HOST.Key})]")
Finally
HOST.DownloadDone(PDownload.SavedPosts)
End Try
End Sub
End Class
End Namespace

View File

@@ -1,200 +0,0 @@
' Copyright (C) 2022 Andy
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
' (at your option) any later version.
'
' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY
Imports PersonalUtilities.Tools
Imports PersonalUtilities.Functions.XML
Imports PersonalUtilities.Functions.XML.Base
Namespace API.Base
Friend Class SiteSettings : Implements IDisposable
Friend Const Header_Twitter_Authorization As String = "authorization"
Friend Const Header_Twitter_Token As String = "x-csrf-token"
Friend ReadOnly Site As Sites
Friend ReadOnly Responser As WEB.Response
Private ReadOnly _Path As XMLValue(Of SFile)
Friend Property Path As SFile
Get
If _Path.IsEmptyString Then _Path.Value = SFile.GetPath($"{Settings.GlobalPath.Value.PathWithSeparator}{Site}")
Return _Path.Value
End Get
Set(ByVal NewFile As SFile)
_Path.Value = NewFile
End Set
End Property
Friend ReadOnly Property InstaHash As XMLValue(Of String)
Friend ReadOnly Property InstaHashUpdateRequired As XMLValue(Of Boolean)
Friend ReadOnly Property InstagramDownloadingErrorDate As XMLValue(Of Date)
Friend Property InstagramLastApplyingValue As Integer? = Nothing
Friend ReadOnly Property InstagramReadyForDownload As Boolean
Get
With InstagramDownloadingErrorDate
If .ValueF.Exists Then
Return .ValueF.Value.AddMinutes(If(InstagramLastApplyingValue, 10)) < Now
Else
Return True
End If
End With
End Get
End Property
Friend Property InstagramTooManyRequestsReadyForCatch As Boolean = True
Friend Sub InstagramTooManyRequests(ByVal Catched As Boolean)
With InstagramDownloadingErrorDate
If Catched Then
If Not .ValueF.Exists Then
.Value = Now
If InstagramTooManyRequestsReadyForCatch Then
InstagramLastApplyingValue = If(InstagramLastApplyingValue, 0) + 10
InstagramTooManyRequestsReadyForCatch = False
MyMainLOG = $"Instagram downloading error: too many requests. Try again after {If(InstagramLastApplyingValue, 10)} minutes..."
End If
End If
Else
.ValueF = Nothing
InstagramLastApplyingValue = Nothing
End If
End With
End Sub
Friend ReadOnly Property Temporary As XMLValue(Of Boolean)
Friend ReadOnly Property DownloadImages As XMLValue(Of Boolean)
Friend ReadOnly Property DownloadVideos As XMLValue(Of Boolean)
Friend ReadOnly Property GetUserMediaOnly As XMLValue(Of Boolean)
Friend ReadOnly Property SavedPostsUserName As XMLValue(Of String)
Private ReadOnly SettingsFile As SFile
Friend Sub New(ByVal s As Sites, ByRef _XML As XmlFile, ByVal GlobalPath As SFile,
ByRef _Temp As XMLValue(Of Boolean), ByRef _Imgs As XMLValue(Of Boolean), ByRef _Vids As XMLValue(Of Boolean))
Site = s
SettingsFile = $"{SettingsFolderName}\Responser_{s}.xml"
Responser = New WEB.Response(SettingsFile)
If SettingsFile.Exists Then
Responser.LoadSettings()
Else
Select Case Site
Case Sites.Twitter
With Responser
.ContentType = "application/json"
.Accept = "*/*"
.CookiesDomain = "twitter.com"
.Decoders.Add(SymbolsConverter.Converters.Unicode)
With .Headers
.Add("sec-ch-ua", " Not;A Brand" & Chr(34) & ";v=" & Chr(34) & "99" & Chr(34) & ", " & Chr(34) &
"Google Chrome" & Chr(34) & ";v=" & Chr(34) & "91" & Chr(34) & ", " & Chr(34) & "Chromium" &
Chr(34) & ";v=" & Chr(34) & "91" & Chr(34))
.Add("sec-ch-ua-mobile", "?0")
.Add("sec-fetch-dest", "empty")
.Add("sec-fetch-mode", "cors")
.Add("sec-fetch-site", "same-origin")
.Add(Header_Twitter_Token, String.Empty)
.Add("x-twitter-active-user", "yes")
.Add("x-twitter-auth-type", "OAuth2Session")
.Add(Header_Twitter_Authorization, String.Empty)
End With
End With
Case Sites.Reddit
Responser.CookiesDomain = "reddit.com"
Responser.Decoders.Add(SymbolsConverter.Converters.Unicode)
Case Sites.Instagram : Responser.CookiesDomain = "instagram.com"
End Select
Responser.SaveSettings()
End If
Dim n() As String = {SettingsCLS.Name_Node_Sites, Site.ToString}
_Path = New XMLValue(Of SFile)("Path", SFile.GetPath($"{GlobalPath.PathWithSeparator}{Site}"), _XML, n, XMLValue(Of SFile).ToFilePath)
_Path.ReplaceByValue("Path", {Site.ToString})
_XML.Remove(Site.ToString)
Temporary = New XMLValue(Of Boolean)
Temporary.SetExtended("Temporary", False, _XML, n)
Temporary.SetDefault(_Temp)
DownloadImages = New XMLValue(Of Boolean)
DownloadImages.SetExtended("DownloadImages", True, _XML, n)
DownloadImages.SetDefault(_Imgs)
DownloadVideos = New XMLValue(Of Boolean)
DownloadVideos.SetExtended("DownloadVideos", True, _XML, n)
DownloadVideos.SetDefault(_Vids)
If Site = Sites.Twitter Then
GetUserMediaOnly = New XMLValue(Of Boolean)("GetUserMediaOnly", True, _XML, n)
GetUserMediaOnly.ReplaceByValue("TwitterDefaultGetUserMedia", n)
Else
GetUserMediaOnly = New XMLValue(Of Boolean)
End If
If Site = Sites.Instagram Then
InstaHash = New XMLValue(Of String)("InstaHash", String.Empty, _XML, n)
InstaHashUpdateRequired = New XMLValue(Of Boolean)("InstaHashUpdateRequired", True, _XML, n)
If (InstaHash.IsEmptyString Or InstaHashUpdateRequired) And Responser.Cookies.ListExists Then GatherInstaHash()
InstagramDownloadingErrorDate = New XMLValue(Of Date) With {.ToStringFunction = Function(ss, vv) AConvert(Of String)(vv, Nothing)}
InstagramDownloadingErrorDate.SetExtended("InstagramDownloadingErrorDate", Now.AddYears(-10), _XML, n)
Else
InstaHash = New XMLValue(Of String)
InstaHashUpdateRequired = New XMLValue(Of Boolean)
End If
If Site = Sites.Reddit Then
SavedPostsUserName = New XMLValue(Of String)("SavedPostsUserName", String.Empty, _XML, n)
Else
SavedPostsUserName = New XMLValue(Of String)
End If
End Sub
Friend Sub Update()
Responser.SaveSettings()
End Sub
Friend Function GatherInstaHash() As Boolean
Try
Dim rs As New RegexStructure("=" & Chr(34) & "([^" & Chr(34) & "]+?ConsumerLibCommons[^" & Chr(34) & "]+?.js)" & Chr(34), 1) With {
.UseTimeOut = True,
.MatchTimeOutSeconds = 10
}
Dim r$ = Responser.GetResponse("https://instagram.com",, EDP.ThrowException)
If Not r.IsEmptyString Then
Dim hStr$ = RegexReplace(r, rs)
If Not hStr.IsEmptyString Then
Do While Left(hStr, 1) = "/" : hStr = Right(hStr, hStr.Length - 1) : Loop
hStr = $"https://instagram.com/{hStr}"
r = Responser.GetResponse(hStr,, EDP.ThrowException)
If Not r.IsEmptyString Then
rs = New RegexStructure("generatePaginationActionCreators.+?.profilePosts.byUserId.get.+?queryId:.([\d\w\S]+?)" & Chr(34), 1) With {
.UseTimeOut = True,
.MatchTimeOutSeconds = 10
}
Dim h$ = RegexReplace(r, rs)
If Not h.IsEmptyString Then
InstaHash.Value = h
InstaHashUpdateRequired.Value = False
Return True
End If
End If
End If
End If
Return False
Catch ex As Exception
InstaHashUpdateRequired.Value = True
InstaHash.Value = String.Empty
Return ErrorsDescriber.Execute(EDP.SendInLog + EDP.ReturnValue, ex, "[SiteSettings.GaterInstaHash]", False)
End Try
End Function
#Region "IDisposable Support"
Private disposedValue As Boolean = False
Protected Overridable Overloads Sub Dispose(ByVal disposing As Boolean)
If Not disposedValue Then
If disposing Then Responser.Dispose()
disposedValue = True
End If
End Sub
Protected Overrides Sub Finalize()
Dispose(False)
MyBase.Finalize()
End Sub
Friend Overloads Sub Dispose() Implements IDisposable.Dispose
Dispose(True)
GC.SuppressFinalize(Me)
End Sub
#End Region
End Class
End Namespace

View File

@@ -0,0 +1,112 @@
' Copyright (C) 2022 Andy
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
' (at your option) any later version.
'
' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY
Imports SCrawler.Plugin
Imports PersonalUtilities.Functions.RegularExpressions
Imports PersonalUtilities.Tools.WEB
Imports Download = SCrawler.Plugin.ISiteSettings.Download
Namespace API.Base
Friend MustInherit Class SiteSettingsBase : Implements ISiteSettings
Friend ReadOnly Property Site As String Implements ISiteSettings.Site
Friend Overridable ReadOnly Property Icon As Icon = Nothing Implements ISiteSettings.Icon
Friend Overridable ReadOnly Property Image As Image = Nothing Implements ISiteSettings.Image
Private Property Logger As ILogProvider = LogConnector Implements ISiteSettings.Logger
Friend Overridable ReadOnly Property Responser As Response
Friend MustOverride Function GetInstance(ByVal What As Download) As IPluginContentProvider Implements ISiteSettings.GetInstance
Friend Sub New(ByVal SiteName As String)
Site = SiteName
End Sub
Friend Sub New(ByVal SiteName As String, ByVal CookiesDomain As String)
Site = SiteName
Responser = New Response($"{SettingsFolderName}\Responser_{Site}.xml")
With Responser
If .File.Exists Then .LoadSettings() Else .CookiesDomain = CookiesDomain : .SaveSettings()
End With
End Sub
#Region "XML"
Friend Overridable Sub Load(ByVal XMLValues As IEnumerable(Of KeyValuePair(Of String, String))) Implements ISiteSettings.Load
End Sub
#End Region
#Region "Initialize"
Friend Overridable Sub BeginInit() Implements ISiteSettings.BeginInit
End Sub
Friend Overridable Sub EndInit() Implements ISiteSettings.EndInit
End Sub
Friend Overridable Sub BeginUpdate() Implements ISiteSettings.BeginUpdate
End Sub
Friend Overridable Sub EndUpdate() Implements ISiteSettings.EndUpdate
End Sub
#End Region
#Region "Before and After Download"
Friend Overridable Sub DownloadStarted(ByVal What As Download) Implements ISiteSettings.DownloadStarted
End Sub
Friend Overridable Sub BeforeStartDownload(ByVal User As Object, ByVal What As Download) Implements ISiteSettings.BeforeStartDownload
End Sub
Friend Overridable Sub AfterDownload(ByVal User As Object, ByVal What As Download) Implements ISiteSettings.AfterDownload
End Sub
Friend Overridable Sub DownloadDone(ByVal What As Download) Implements ISiteSettings.DownloadDone
End Sub
#End Region
#Region "User info"
Protected UrlPatternUser As String = String.Empty
Protected UrlPatternChannel As String = String.Empty
Friend Overridable Function GetUserUrl(ByVal UserName As String, ByVal Channel As Boolean) As String Implements ISiteSettings.GetUserUrl
If Channel Then
If Not UrlPatternChannel.IsEmptyString Then Return String.Format(UrlPatternChannel, UserName)
Else
If Not UrlPatternUser.IsEmptyString Then Return String.Format(UrlPatternUser, UserName)
End If
Return String.Empty
End Function
Protected UserRegex As RParams = Nothing
Friend Overridable Function IsMyUser(ByVal UserURL As String) As ExchangeOptions Implements ISiteSettings.IsMyUser
Try
If Not UserRegex Is Nothing Then
Dim s$ = RegexReplace(UserURL, UserRegex)
If Not s.IsEmptyString Then Return New ExchangeOptions(Site, s)
End If
Return Nothing
Catch ex As Exception
Return ErrorsDescriber.Execute(EDP.SendInLog + EDP.ReturnValue, ex, "[API.Base.SiteSettingsBase.IsMyUser]")
End Try
End Function
Protected ImageVideoContains As String = String.Empty
Friend Overridable Function IsMyImageVideo(ByVal URL As String) As ExchangeOptions Implements ISiteSettings.IsMyImageVideo
If Not ImageVideoContains.IsEmptyString AndAlso URL.Contains(ImageVideoContains) Then
Return New ExchangeOptions With {.Exists = True}
Else
Return Nothing
End If
End Function
Friend Overridable Function GetSpecialData(ByVal URL As String, ByVal Path As String, ByVal AskForPath As Boolean) As IEnumerable(Of PluginUserMedia) Implements ISiteSettings.GetSpecialData
Return Nothing
End Function
Friend Overridable Function GetSpecialDataF(ByVal URL As String) As IEnumerable(Of UserMedia)
Return Nothing
End Function
#End Region
#Region "Ready, Available"
Friend Overridable Function Available(ByVal What As Download) As Boolean Implements ISiteSettings.Available
Return True
End Function
Friend Overridable Function ReadyToDownload(ByVal What As Download) As Boolean Implements ISiteSettings.ReadyToDownload
Return True
End Function
#End Region
Friend Overridable Sub Update() Implements ISiteSettings.Update
If Not Responser Is Nothing Then Responser.SaveSettings()
End Sub
Friend Overridable Sub Reset() Implements ISiteSettings.Reset
End Sub
Friend Overridable Sub UserOptions(ByRef Options As Object, ByVal OpenForm As Boolean) Implements ISiteSettings.UserOptions
Options = Nothing
End Sub
Friend Overridable Sub OpenSettingsForm() Implements ISiteSettings.OpenSettingsForm
End Sub
End Class
End Namespace

View File

@@ -18,8 +18,8 @@ Namespace API.Base
GIF = 50 GIF = 50
m3u8 = 100 m3u8 = 100
End Enum End Enum
Friend Enum States : Unknown : Tried : Downloaded : Skipped : End Enum Friend Enum States As Integer : Unknown = 0 : Tried = 1 : Downloaded = 2 : Skipped = 3 : End Enum
Friend Type As Types Friend [Type] As Types
Friend URL_BASE As String Friend URL_BASE As String
Friend URL As String Friend URL As String
Friend MD5 As String Friend MD5 As String
@@ -27,6 +27,11 @@ Namespace API.Base
Friend Post As UserPost Friend Post As UserPost
Friend PictureOption As String Friend PictureOption As String
Friend State As States Friend State As States
''' <summary>
''' SomeFolder<br/>
''' SomeFolder\SomeFolder2
''' </summary>
Friend SpecialFolder As String
Friend Sub New(ByVal _URL As String) Friend Sub New(ByVal _URL As String)
URL = _URL URL = _URL
URL_BASE = _URL URL_BASE = _URL
@@ -35,7 +40,18 @@ Namespace API.Base
End Sub End Sub
Friend Sub New(ByVal _URL As String, ByVal _Type As Types) Friend Sub New(ByVal _URL As String, ByVal _Type As Types)
Me.New(_URL) Me.New(_URL)
Type = _Type [Type] = _Type
End Sub
Friend Sub New(ByVal m As Plugin.PluginUserMedia)
If Not IsNothing(m) Then
[Type] = m.ContentType
URL = m.URL
MD5 = m.MD5
File = m.File
Post = New UserPost With {.ID = m.PostID, .[Date] = m.PostDate}
State = m.DownloadState
SpecialFolder = m.SpecialFolder
End If
End Sub End Sub
Public Shared Widening Operator CType(ByVal _URL As String) As UserMedia Public Shared Widening Operator CType(ByVal _URL As String) As UserMedia
Return New UserMedia(_URL) Return New UserMedia(_URL)
@@ -46,6 +62,18 @@ Namespace API.Base
Public Overrides Function ToString() As String Public Overrides Function ToString() As String
Return URL Return URL
End Function End Function
Friend Function PluginUserMedia() As Plugin.PluginUserMedia
Return New Plugin.PluginUserMedia With {
.ContentType = Type,
.DownloadState = State,
.File = File,
.MD5 = MD5,
.URL = URL,
.SpecialFolder = SpecialFolder,
.PostID = Post.ID,
.PostDate = Post.Date
}
End Function
Friend Overloads Function Equals(ByVal Other As UserMedia) As Boolean Implements IEquatable(Of UserMedia).Equals Friend Overloads Function Equals(ByVal Other As UserMedia) As Boolean Implements IEquatable(Of UserMedia).Equals
Return URL = Other.URL Return URL = Other.URL
End Function End Function
@@ -85,5 +113,21 @@ Namespace API.Base
Return v Return v
End Function End Function
End Structure End Structure
Friend Structure Sizes : Implements IComparable(Of Sizes)
Friend Value As Integer
Friend Data As String
Friend ReadOnly HasError As Boolean
Friend Sub New(ByVal _Value As String, ByVal _Data As String)
Try
Value = _Value
Data = _Data
Catch ex As Exception
HasError = True
End Try
End Sub
Friend Function CompareTo(ByVal Other As Sizes) As Integer Implements IComparable(Of Sizes).CompareTo
Return Value.CompareTo(Other.Value) * -1
End Function
End Structure
End Module End Module
End Namespace End Namespace

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,35 @@
' Copyright (C) 2022 Andy
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
' (at your option) any later version.
'
' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY
Imports PersonalUtilities.Functions.RegularExpressions
Imports System.Net
Imports SCrawler.API.Base
Namespace API.Gfycat
Friend NotInheritable Class Envir
Private Sub New()
End Sub
Friend Shared Function GetVideo(ByVal URL As String) As String
Try
Dim r$
Using w As New WebClient : r = w.DownloadString(URL) : End Using
If Not r.IsEmptyString Then Return RegexReplace(r, RParams.DMS("contentUrl.:.(http.?://[^""]+?\.mp4)", 1)) Else Return String.Empty
Catch ex As Exception
Dim e As EDP = EDP.ReturnValue
If TypeOf ex Is WebException Then
Dim obj As HttpWebResponse = TryCast(DirectCast(ex, WebException).Response, HttpWebResponse)
If Not If(obj?.StatusCode, HttpStatusCode.OK) = HttpStatusCode.NotFound Then e += EDP.SendInLog
End If
Return ErrorsDescriber.Execute(e, ex, $"[API.Gfycat.Envir.GetVideo({URL})]", String.Empty)
End Try
End Function
Friend Shared Function GetVideoInfo(ByVal URL As String) As IEnumerable(Of UserMedia)
Dim u$ = GetVideo(URL)
Return If(u.IsEmptyString, Nothing, {New UserMedia(u, UserMedia.Types.Video)})
End Function
End Class
End Namespace

View File

@@ -7,19 +7,21 @@
' This program is distributed in the hope that it will be useful, ' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY ' but WITHOUT ANY WARRANTY
Imports PersonalUtilities.Functions.XML Imports PersonalUtilities.Functions.XML
Imports PersonalUtilities.Functions.RegularExpressions
Imports PersonalUtilities.Tools.WebDocuments.JSON Imports PersonalUtilities.Tools.WebDocuments.JSON
Imports System.Net Imports System.Net
Imports SCrawler.API.Imgur.Declarations Imports SCrawler.API.Imgur.Declarations
Namespace API.Imgur.Declarations Imports SCrawler.API.Base
Namespace API.Imgur
Namespace Declarations
Friend Module Imgur_Declarations Friend Module Imgur_Declarations
Friend ReadOnly PostRegex As New RegexStructure("/([\w\d]+?)(|\.[\w]{0,4})\Z", 1) Friend ReadOnly PostRegex As RParams = RParams.DMS("/([\w\d]+?)(|\.[\w]{0,4})\Z", 1)
End Module End Module
End Namespace End Namespace
Namespace API.Imgur
Friend NotInheritable Class Envir Friend NotInheritable Class Envir
Private Sub New() Private Sub New()
End Sub End Sub
Friend Shared Function GetGallery(ByVal URL As String) As List(Of String) Friend Shared Function GetGallery(ByVal URL As String, Optional ByVal e As ErrorsDescriber = Nothing) As List(Of String)
Try Try
If Not Settings.ImgurClientID.IsEmptyString And Not URL.IsEmptyString Then If Not Settings.ImgurClientID.IsEmptyString And Not URL.IsEmptyString Then
Dim __url$ = RegexReplace(URL, PostRegex) Dim __url$ = RegexReplace(URL, PostRegex)
@@ -45,10 +47,10 @@ Namespace API.Imgur
End If End If
Return Nothing Return Nothing
Catch ex As Exception Catch ex As Exception
Return ErrorsDescriber.Execute(EDP.ReturnValue + EDP.SendInLog, ex, $"[API.Imgur.Envir.GetGallery({URL})]", Nothing) Return DownloadingException(ex, $"[API.Imgur.Envir.GetGallery({URL})]", Nothing, e)
End Try End Try
End Function End Function
Friend Shared Function GetImage(ByVal URL As String) As String Friend Shared Function GetImage(ByVal URL As String, Optional ByVal e As ErrorsDescriber = Nothing) As String
Try Try
If Not Settings.ImgurClientID.IsEmptyString And Not URL.IsEmptyString Then If Not Settings.ImgurClientID.IsEmptyString And Not URL.IsEmptyString Then
Dim __url$ = RegexReplace(URL, PostRegex) Dim __url$ = RegexReplace(URL, PostRegex)
@@ -62,8 +64,39 @@ Namespace API.Imgur
End If End If
Return String.Empty Return String.Empty
Catch ex As Exception Catch ex As Exception
Return ErrorsDescriber.Execute(EDP.ReturnValue + EDP.SendInLog, ex, $"[API.Imgur.Envir.GetImage({URL})]", String.Empty) Return DownloadingException(ex, $"[API.Imgur.Envir.GetImage({URL})]", String.Empty, e)
End Try End Try
End Function End Function
Friend Shared Function GetVideoInfo(ByVal URL As String) As IEnumerable(Of UserMedia)
Try
If Not URL.IsEmptyString AndAlso URL.ToLower.Contains("imgur") AndAlso Not Settings.ImgurClientID.IsEmptyString Then
Dim img$ = GetImage(URL, EDP.ReturnValue)
If Not img.IsEmptyString Then
Return {New UserMedia(img)}
Else
Return GetGallery(URL, EDP.ReturnValue).ListIfNothing.Select(Function(u) New UserMedia(u))
End If
End If
Return Nothing
Catch ex As Exception
Return ErrorsDescriber.Execute(EDP.ShowMainMsg + EDP.SendInLog, ex, "Imgur standalone downloader: fetch media error")
End Try
End Function
Private Shared Function DownloadingException(ByVal ex As Exception, ByVal Message As String,
ByVal NullArg As Object, ByVal e As ErrorsDescriber) As Object
If TypeOf ex Is WebException Then
Dim obj As HttpWebResponse = TryCast(DirectCast(ex, WebException).Response, HttpWebResponse)
If Not obj Is Nothing Then
If obj.StatusCode = HttpStatusCode.NotFound Then
Return NullArg
ElseIf obj.StatusCode = HttpStatusCode.Unauthorized Then
MyMainLOG = "Imgur credentials expired"
Return NullArg
End If
End If
End If
If Not e.Exists Then e = New ErrorsDescriber(EDP.ReturnValue + EDP.SendInLog)
Return ErrorsDescriber.Execute(e, ex, Message, NullArg)
End Function
End Class End Class
End Namespace End Namespace

View File

@@ -0,0 +1,36 @@
' Copyright (C) 2022 Andy
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
' (at your option) any later version.
'
' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY
Imports Sections = SCrawler.API.Instagram.UserData.Sections
Namespace API.Instagram
Friend Class AuthNullException : Inherits ArgumentNullException
Public Overrides ReadOnly Property ParamName As String
Public Overrides ReadOnly Property Message As String
Friend Sub New(ByVal s As Sections, ByVal IsSavedPosts As Boolean)
If IsSavedPosts Then
ParamName = "HashSavedPosts"
ElseIf s = Sections.Timeline Then
ParamName = "Hash"
Else
ParamName = "IG_APP_ID, IG_WWW_CLAIM"
End If
Message = $"Instagram auth for [{s}] is not set"
End Sub
Friend Shared Sub ThrowIfNull(ByVal s As Sections, ByVal IsSavedPosts As Boolean, ByVal Host As SiteSettings)
Dim b As Boolean = False
If IsSavedPosts Then
If Not ACheck(Host.HashSavedPosts.Value) Then b = True
ElseIf s = Sections.Timeline Then
If Not ACheck(Host.Hash.Value) Then Host.HashUpdateRequired.Value = True : b = True
Else
If Not Host.StoriesAndTaggedReady Then b = True
End If
If b Then Throw New AuthNullException(s, IsSavedPosts)
End Sub
End Class
End Namespace

View File

@@ -6,9 +6,11 @@
' '
' This program is distributed in the hope that it will be useful, ' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY ' but WITHOUT ANY WARRANTY
Imports PersonalUtilities.Functions.RegularExpressions
Namespace API.Instagram Namespace API.Instagram
Friend Module Declarations Friend Module Declarations
Friend ReadOnly FilesPattern As New RegexStructure("[^\./]+?\.\w+", True, False, 2,,,, String.Empty, EDP.ReturnValue) Friend Const InstagramSite As String = "Instagram"
Friend ReadOnly FilesPattern As RParams = RParams.DMS(".+?([^/\?]+?\.[\w\d]{3,4})(?=(\?|\Z))", 1, EDP.ReturnValue)
Friend ReadOnly Property DateProvider As New JsonDate Friend ReadOnly Property DateProvider As New JsonDate
Friend Class JsonDate : Implements ICustomProvider Friend Class JsonDate : Implements ICustomProvider
Friend Function Convert(ByVal Value As Object, ByVal DestinationType As Type, ByVal Provider As IFormatProvider, Friend Function Convert(ByVal Value As Object, ByVal DestinationType As Type, ByVal Provider As IFormatProvider,

View File

@@ -0,0 +1,23 @@
' Copyright (C) 2022 Andy
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
' (at your option) any later version.
'
' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY
Imports SCrawler.Plugin
Namespace API.Instagram
Friend Class EditorExchangeOptions
Friend Property GetStories As Boolean
Friend Property GetTagged As Boolean
Private ReadOnly Property MySiteSettings As SiteSettings
Friend Sub New(ByVal h As ISiteSettings)
MySiteSettings = DirectCast(h, SiteSettings)
With MySiteSettings
GetStories = CBool(.GetStories.Value)
GetTagged = CBool(.GetTagged.Value)
End With
End Sub
End Class
End Namespace

View File

@@ -0,0 +1,15 @@
' Copyright (C) 2022 Andy
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
' (at your option) any later version.
'
' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY
Namespace API.Instagram
Friend Class ExitException : Inherits Exception
Friend Sub New(ByRef CompleteArg As Boolean)
CompleteArg = True
End Sub
End Class
End Namespace

View File

@@ -0,0 +1,120 @@
' Copyright (C) 2022 Andy
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
' (at your option) any later version.
'
' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY
Namespace API.Instagram
<Global.Microsoft.VisualBasic.CompilerServices.DesignerGenerated()>
Partial Friend Class OptionsForm : Inherits System.Windows.Forms.Form
<System.Diagnostics.DebuggerNonUserCode()>
Protected Overrides Sub Dispose(ByVal disposing As Boolean)
Try
If disposing AndAlso components IsNot Nothing Then
components.Dispose()
End If
Finally
MyBase.Dispose(disposing)
End Try
End Sub
Private components As System.ComponentModel.IContainer
<System.Diagnostics.DebuggerStepThrough()>
Private Sub InitializeComponent()
Dim CONTAINER_MAIN As System.Windows.Forms.ToolStripContainer
Dim TP_MAIN As System.Windows.Forms.TableLayoutPanel
Me.CH_GET_STORIES = New System.Windows.Forms.CheckBox()
Me.CH_GET_TAGGED = New System.Windows.Forms.CheckBox()
CONTAINER_MAIN = New System.Windows.Forms.ToolStripContainer()
TP_MAIN = New System.Windows.Forms.TableLayoutPanel()
CONTAINER_MAIN.ContentPanel.SuspendLayout()
CONTAINER_MAIN.SuspendLayout()
TP_MAIN.SuspendLayout()
Me.SuspendLayout()
'
'CONTAINER_MAIN
'
'
'CONTAINER_MAIN.ContentPanel
'
CONTAINER_MAIN.ContentPanel.Controls.Add(TP_MAIN)
CONTAINER_MAIN.ContentPanel.Size = New System.Drawing.Size(260, 53)
CONTAINER_MAIN.Dock = System.Windows.Forms.DockStyle.Fill
CONTAINER_MAIN.LeftToolStripPanelVisible = False
CONTAINER_MAIN.Location = New System.Drawing.Point(0, 0)
CONTAINER_MAIN.Name = "CONTAINER_MAIN"
CONTAINER_MAIN.RightToolStripPanelVisible = False
CONTAINER_MAIN.Size = New System.Drawing.Size(260, 78)
CONTAINER_MAIN.TabIndex = 0
CONTAINER_MAIN.TopToolStripPanelVisible = False
'
'TP_MAIN
'
TP_MAIN.CellBorderStyle = System.Windows.Forms.TableLayoutPanelCellBorderStyle.[Single]
TP_MAIN.ColumnCount = 1
TP_MAIN.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Percent, 100.0!))
TP_MAIN.Controls.Add(Me.CH_GET_STORIES, 0, 0)
TP_MAIN.Controls.Add(Me.CH_GET_TAGGED, 0, 1)
TP_MAIN.Dock = System.Windows.Forms.DockStyle.Fill
TP_MAIN.Location = New System.Drawing.Point(0, 0)
TP_MAIN.Name = "TP_MAIN"
TP_MAIN.RowCount = 3
TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!))
TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!))
TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Percent, 100.0!))
TP_MAIN.Size = New System.Drawing.Size(260, 53)
TP_MAIN.TabIndex = 0
'
'CH_GET_STORIES
'
Me.CH_GET_STORIES.AutoSize = True
Me.CH_GET_STORIES.Dock = System.Windows.Forms.DockStyle.Fill
Me.CH_GET_STORIES.Location = New System.Drawing.Point(4, 4)
Me.CH_GET_STORIES.Name = "CH_GET_STORIES"
Me.CH_GET_STORIES.Size = New System.Drawing.Size(252, 19)
Me.CH_GET_STORIES.TabIndex = 0
Me.CH_GET_STORIES.Text = "Get stories"
Me.CH_GET_STORIES.UseVisualStyleBackColor = True
'
'CH_GET_TAGGED
'
Me.CH_GET_TAGGED.AutoSize = True
Me.CH_GET_TAGGED.Dock = System.Windows.Forms.DockStyle.Fill
Me.CH_GET_TAGGED.Location = New System.Drawing.Point(4, 30)
Me.CH_GET_TAGGED.Name = "CH_GET_TAGGED"
Me.CH_GET_TAGGED.Size = New System.Drawing.Size(252, 19)
Me.CH_GET_TAGGED.TabIndex = 1
Me.CH_GET_TAGGED.Text = "Get tagged data"
Me.CH_GET_TAGGED.UseVisualStyleBackColor = True
'
'OptionsForm
'
Me.AutoScaleDimensions = New System.Drawing.SizeF(6.0!, 13.0!)
Me.AutoScaleMode = System.Windows.Forms.AutoScaleMode.Font
Me.ClientSize = New System.Drawing.Size(260, 78)
Me.Controls.Add(CONTAINER_MAIN)
Me.FormBorderStyle = System.Windows.Forms.FormBorderStyle.FixedSingle
Me.KeyPreview = True
Me.MaximizeBox = False
Me.MaximumSize = New System.Drawing.Size(276, 117)
Me.MinimizeBox = False
Me.MinimumSize = New System.Drawing.Size(276, 117)
Me.Name = "OptionsForm"
Me.ShowIcon = False
Me.ShowInTaskbar = False
Me.SizeGripStyle = System.Windows.Forms.SizeGripStyle.Hide
Me.Text = "Options"
CONTAINER_MAIN.ContentPanel.ResumeLayout(False)
CONTAINER_MAIN.ResumeLayout(False)
CONTAINER_MAIN.PerformLayout()
TP_MAIN.ResumeLayout(False)
TP_MAIN.PerformLayout()
Me.ResumeLayout(False)
End Sub
Private WithEvents CH_GET_STORIES As CheckBox
Private WithEvents CH_GET_TAGGED As CheckBox
End Class
End Namespace

View File

@@ -117,26 +117,10 @@
<resheader name="writer"> <resheader name="writer">
<value>System.Resources.ResXResourceWriter, System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value> <value>System.Resources.ResXResourceWriter, System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
</resheader> </resheader>
<metadata name="ToolbarTOP.TrayLocation" type="System.Drawing.Point, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a"> <metadata name="CONTAINER_MAIN.GenerateMember" type="System.Boolean, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
<value>17, 17</value> <value>False</value>
</metadata> </metadata>
<assembly alias="System.Drawing" name="System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" /> <metadata name="TP_MAIN.GenerateMember" type="System.Boolean, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
<data name="MENU_VIEW.Image" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64"> <value>False</value>
<value>
iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8
YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAAIDSURBVDhPpZLrS5NhGMb3j4SWh0oRQVExD4gonkDpg4hG
YKxG6WBogkMZKgPNCEVJFBGdGETEvgwyO9DJE5syZw3PIlPEE9pgBCLZ5XvdMB8Ew8gXbl54nuf63dd9
0OGSnwCahxbPRNPAPMw9Xpg6ZmF46kZZ0xSKzJPIrhpDWsVnpBhGkKx3nAX8Pv7z1zg8OoY/cITdn4fw
bf/C0kYAN3Ma/w3gWfZL5kzTKBxjWyK2DftwI9tyMYCZKXbNHaD91bLYJrDXsYbrWfUKwJrPE9M2M1Oc
VzOOpHI7Jr376Hi9ogHqFIANO0/MmmmbmSmm9a8ze+I4MrNWAdjtoJgWcx+PSzg166yZZ8xM8XvXDix9
c4jIqFYAjoriBV9AhEPv1mH/sonogha0afbZMMZz+yreTGyhpusHwtNNCsA5U1zS4BLxzJIfg299qO32
Ir7UJtZfftyATqeT+8o2D8JSjQrAJblrncYL7ZJ2+bfaFnC/1S1NjL3diRat7qrO7wLRP3HjWsojBeCo
mDEo5mNjuweFGvjWg2EBhCbpkW78htSHHwRyNdmgAFzPEee2iFkzayy2OLXzT4gr6UdUnlXrullsxxQ+
kx0g8BTA3aZlButjSTyjODq/WcQcW/B/Je4OQhLvKQDnzN1mp0nnkvAhR8VuMzNrpm1mpjgkoVwB/v8D
TgDQASA1MVpwzwAAAABJRU5ErkJggg==
</value>
</data>
<metadata name="ToolbarBOTTOM.TrayLocation" type="System.Drawing.Point, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
<value>133, 17</value>
</metadata> </metadata>
</root> </root>

View File

@@ -0,0 +1,44 @@
' Copyright (C) 2022 Andy
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
' (at your option) any later version.
'
' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY
Imports PersonalUtilities.Forms
Imports PersonalUtilities.Forms.Toolbars
Namespace API.Instagram
Friend Class OptionsForm : Implements IOkCancelToolbar
Private ReadOnly MyDefs As DefaultFormProps
Private ReadOnly Property MyExchangeOptions As EditorExchangeOptions
Friend Sub New(ByRef ExchangeOptions As EditorExchangeOptions)
InitializeComponent()
MyExchangeOptions = ExchangeOptions
MyDefs = New DefaultFormProps
End Sub
Private Sub OptionsForm_Load(sender As Object, e As EventArgs) Handles Me.Load
With MyDefs
.MyViewInitialize(Me, Settings.Design, True)
.AddOkCancelToolbar()
.DelegateClosingChecker()
.AppendDetectors()
With MyExchangeOptions
CH_GET_STORIES.Checked = .GetStories
CH_GET_TAGGED.Checked = .GetTagged
End With
.EndLoaderOperations()
End With
End Sub
Private Sub ToolbarBttOK() Implements IOkCancelToolbar.ToolbarBttOK
With MyExchangeOptions
.GetStories = CH_GET_STORIES.Checked
.GetTagged = CH_GET_TAGGED.Checked
End With
MyDefs.CloseForm()
End Sub
Private Sub ToolbarBttCancel() Implements IOkCancelToolbar.ToolbarBttCancel
MyDefs.CloseForm(DialogResult.Cancel)
End Sub
End Class
End Namespace

View File

@@ -0,0 +1,315 @@
' Copyright (C) 2022 Andy
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
' (at your option) any later version.
'
' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY
Imports SCrawler.API.Base
Imports SCrawler.Plugin
Imports SCrawler.Plugin.Attributes
Imports PersonalUtilities.Tools
Imports PersonalUtilities.Functions.XML
Imports PersonalUtilities.Functions.XML.Base
Imports PersonalUtilities.Functions.RegularExpressions
Imports Download = SCrawler.Plugin.ISiteSettings.Download
Namespace API.Instagram
<Manifest("AndyProgram_Instagram"), UseClassAsIs, SeparatedTasks(1), SavedPosts, SpecialForm(False)>
Friend Class SiteSettings : Inherits SiteSettingsBase
#Region "Interface Declarations"
Friend Overrides ReadOnly Property Icon As Icon
Get
Return My.Resources.InstagramIcon
End Get
End Property
Friend Overrides ReadOnly Property Image As Image
Get
Return My.Resources.InstagramPic76
End Get
End Property
#End Region
#Region "Providers"
Private Class TimersChecker : Implements ICustomProvider
Private ReadOnly _LowestValue As Integer
Friend Sub New(ByVal LowestValue As Integer)
_LowestValue = LowestValue
End Sub
Private Function Convert(ByVal Value As Object, ByVal DestinationType As Type, ByVal Provider As IFormatProvider,
Optional ByVal NothingArg As Object = Nothing, Optional ByVal e As ErrorsDescriber = Nothing) As Object Implements ICustomProvider.Convert
If ACheck(Of Integer)(Value) AndAlso CInt(Value) >= _LowestValue Then
Return Value
Else
Return Nothing
End If
End Function
Private Function GetFormat(ByVal FormatType As Type) As Object Implements IFormatProvider.GetFormat
Throw New NotImplementedException()
End Function
End Class
#End Region
#Region "Authorization properties"
<PropertyOption(ControlText:="Hash", ControlToolTip:="Instagram session hash", IsAuth:=True), PXML("InstaHash"), ControlNumber(0)>
Friend ReadOnly Property Hash As PropertyValue
<PropertyOption(ControlText:="Hash 2", ControlToolTip:="Instagram session hash for saved posts", IsAuth:=True), PXML("InstaHashSavedPosts"), ControlNumber(1)>
Friend ReadOnly Property HashSavedPosts As PropertyValue
<PropertyOption(ControlText:="x-ig-app-id", IsAuth:=True), ControlNumber(2)>
Friend Property IG_APP_ID As PropertyValue
<PropertyOption(ControlText:="x-ig-www-claim", IsAuth:=True), ControlNumber(3)>
Friend Property IG_WWW_CLAIM As PropertyValue
<PropertyOption(ControlText:="Saved posts user", IsAuth:=True), PXML("SavedPostsUserName"), ControlNumber(4)>
Friend ReadOnly Property SavedPostsUserName As PropertyValue
Friend ReadOnly Property StoriesAndTaggedReady As Boolean
Get
Return ACheck(IG_APP_ID.Value) And ACheck(IG_WWW_CLAIM.Value)
End Get
End Property
#End Region
#Region "Download properties"
Friend ReadOnly Property HashUpdateRequired As XMLValue(Of Boolean)
<PropertyOption(ControlText:="Request timer", AllowNull:=False), PXML("RequestsWaitTimer"), ControlNumber(5)>
Friend ReadOnly Property RequestsWaitTimer As PropertyValue
<Provider(NameOf(RequestsWaitTimer), FieldsChecker:=True)>
Private ReadOnly Property RequestsWaitTimerProvider As IFormatProvider
<PropertyOption(ControlText:="Request timer counter", AllowNull:=False, LeftOffset:=120), PXML("RequestsWaitTimerTaskCount"), ControlNumber(6)>
Friend ReadOnly Property RequestsWaitTimerTaskCount As PropertyValue
<Provider(NameOf(RequestsWaitTimerTaskCount), FieldsChecker:=True)>
Private ReadOnly Property RequestsWaitTimerTaskCountProvider As IFormatProvider
<PropertyOption(ControlText:="Posts limit timer", AllowNull:=False), PXML("SleepTimerOnPostsLimit"), ControlNumber(7)>
Friend ReadOnly Property SleepTimerOnPostsLimit As PropertyValue
<Provider(NameOf(SleepTimerOnPostsLimit), FieldsChecker:=True)>
Private ReadOnly Property SleepTimerOnPostsLimitProvider As IFormatProvider
<PropertyOption(ControlText:="Get stories"), PXML, ControlNumber(8)>
Friend ReadOnly Property GetStories As PropertyValue
<PropertyOption(ControlText:="Get tagged photos"), PXML, ControlNumber(9)>
Friend ReadOnly Property GetTagged As PropertyValue
#End Region
#Region "429 bypass"
Friend ReadOnly Property DownloadingErrorDate As XMLValue(Of Date)
Friend Property LastApplyingValue As Integer? = Nothing
Friend ReadOnly Property ReadyForDownload As Boolean
Get
With DownloadingErrorDate
If .ValueF.Exists Then
Return .ValueF.Value.AddMinutes(If(LastApplyingValue, 10)) < Now
Else
Return True
End If
End With
End Get
End Property
Friend ReadOnly Property LastDownloadDate As XMLValue(Of Date)
Friend ReadOnly Property LastRequestsCount As XMLValue(Of Integer)
Private TooManyRequestsReadyForCatch As Boolean = True
Friend Function GetWaitDate() As Date
With DownloadingErrorDate
If .ValueF.Exists Then
Return .ValueF.Value.AddMinutes(If(LastApplyingValue, 10))
Else
Return Now
End If
End With
End Function
Friend Sub TooManyRequests(ByVal Catched As Boolean)
With DownloadingErrorDate
If Catched Then
If Not .ValueF.Exists Then
.Value = Now
If TooManyRequestsReadyForCatch Then
LastApplyingValue = If(LastApplyingValue, 0) + 10
TooManyRequestsReadyForCatch = False
MyMainLOG = $"Instagram downloading error: too many requests. Try again after {If(LastApplyingValue, 10)} minutes..."
End If
End If
Else
.ValueF = Nothing
LastApplyingValue = Nothing
TooManyRequestsReadyForCatch = True
End If
End With
End Sub
#End Region
Friend Overrides ReadOnly Property Responser As WEB.Response
Friend Sub New(ByRef _XML As XmlFile, ByVal GlobalPath As SFile)
MyBase.New(InstagramSite)
Responser = New WEB.Response($"{SettingsFolderName}\Responser_{Site}.xml")
Dim app_id$ = String.Empty
Dim www_claim$ = String.Empty
With Responser
If .File.Exists Then
.LoadSettings()
With .Headers
If .ContainsKey(Header_IG_APP_ID) Then app_id = .Item(Header_IG_APP_ID)
If .ContainsKey(Header_IG_WWW_CLAIM) Then www_claim = .Item(Header_IG_WWW_CLAIM)
End With
Else
.CookiesDomain = "instagram.com"
.SaveSettings()
End If
End With
Dim n() As String = {SettingsCLS.Name_Node_Sites, Site.ToString}
SavedPostsUserName = New PropertyValue(String.Empty, GetType(String))
HashUpdateRequired = New XMLValue(Of Boolean)("InstaHashUpdateRequired", True, _XML, n)
Hash = New PropertyValue(String.Empty, GetType(String))
HashSavedPosts = New PropertyValue(String.Empty, GetType(String))
IG_APP_ID = New PropertyValue(app_id, GetType(String), Sub(v) ChangeResponserFields(NameOf(IG_APP_ID), v))
IG_WWW_CLAIM = New PropertyValue(www_claim, GetType(String), Sub(v) ChangeResponserFields(NameOf(IG_WWW_CLAIM), v))
RequestsWaitTimer = New PropertyValue(1000)
RequestsWaitTimerProvider = New TimersChecker(100)
RequestsWaitTimerTaskCount = New PropertyValue(1)
RequestsWaitTimerTaskCountProvider = New TimersChecker(1)
SleepTimerOnPostsLimit = New PropertyValue(60000)
SleepTimerOnPostsLimitProvider = New TimersChecker(10000)
GetStories = New PropertyValue(False)
GetTagged = New PropertyValue(False)
DownloadingErrorDate = New XMLValue(Of Date) With {
.Provider = New XMLValueConversionProvider(Function(ss, vv) AConvert(Of String)(vv, AModes.Var, Nothing))}
DownloadingErrorDate.SetExtended("InstagramDownloadingErrorDate", Now.AddYears(-10), _XML, n)
LastDownloadDate = New XMLValue(Of Date)("LastDownloadDate", Now.AddDays(-1), _XML, n)
LastRequestsCount = New XMLValue(Of Integer)("LastRequestsCount", 0, _XML, n)
UrlPatternUser = "https://www.instagram.com/{0}/"
UserRegex = RParams.DMS("[htps:/]{7,8}.*?instagram.com/([^/]+)", 1)
ImageVideoContains = "instagram.com"
End Sub
Friend Overrides Function GetInstance(ByVal What As Download) As IPluginContentProvider
Select Case What
Case Download.Main : Return New UserData
Case Download.SavedPosts
Dim u As New UserData
DirectCast(u, UserDataBase).User = New UserInfo With {.Name = CStr(AConvert(Of String)(SavedPostsUserName.Value, String.Empty))}
Return u
End Select
Return Nothing
End Function
Private Const Header_IG_APP_ID As String = "x-ig-app-id"
Private Const Header_IG_WWW_CLAIM As String = "x-ig-www-claim"
Private Sub ChangeResponserFields(ByVal PropName As String, ByVal Value As Object)
If Not PropName.IsEmptyString Then
Dim f$ = String.Empty
Select Case PropName
Case NameOf(IG_APP_ID) : f = Header_IG_APP_ID
Case NameOf(IG_WWW_CLAIM) : f = Header_IG_WWW_CLAIM
End Select
If Not f.IsEmptyString Then
If Responser.Headers.Count > 0 AndAlso Responser.Headers.ContainsKey(f) Then Responser.Headers.Remove(f)
If Not CStr(Value).IsEmptyString Then Responser.Headers.Add(f, CStr(Value))
Responser.SaveSettings()
End If
End If
End Sub
<PropertiesDataChecker({NameOf(Hash), NameOf(HashSavedPosts)})>
Private Function CheckHashControls(ByVal p As IEnumerable(Of PropertyData)) As Boolean
If p.ListExists(2) Then
Dim h$ = String.Empty
Dim hsp$ = String.Empty
For Each pp As PropertyData In p
Select Case pp.Name
Case NameOf(Hash) : h = AConvert(Of String)(pp.Value, String.Empty)
Case NameOf(HashSavedPosts) : hsp = AConvert(Of String)(pp.Value, String.Empty)
End Select
Next
If h.IsEmptyString And hsp.IsEmptyString Then
Return True
Else
If h = hsp Then
MsgBoxE({"InstaHash for saved posts must be different from InstaHash!", "InstaHash are equal"}, vbCritical)
Return False
Else
Return True
End If
End If
Else
Return False
End If
End Function
Friend Overrides Sub BeginInit()
End Sub
Friend Overrides Sub EndInit()
If (CStr(Hash.Value).IsEmptyString Or HashUpdateRequired) AndAlso Responser.Cookies.ListExists Then GatherInstaHash()
End Sub
Friend Overrides Function ReadyToDownload(ByVal What As Download) As Boolean
Return ActiveJobs < 2 AndAlso ReadyForDownload
End Function
#Region "Downloading"
Private ActiveJobs As Integer = 0
Private _NextWNM As UserData.WNM = UserData.WNM.Notify
Friend Overrides Sub DownloadStarted(ByVal What As Download)
If CStr(Hash.Value).IsEmptyString Or HashUpdateRequired Then GatherInstaHash()
ActiveJobs += 1
End Sub
Friend Overrides Sub BeforeStartDownload(ByVal User As Object, ByVal What As Download)
With DirectCast(User, UserData)
If What = Download.Main Then .WaitNotificationMode = _NextWNM
If LastDownloadDate.Value.AddMinutes(60) > Now Then
.RequestsCount = LastRequestsCount
Else
LastRequestsCount.Value = 0
.RequestsCount = 0
End If
End With
End Sub
Friend Overrides Sub AfterDownload(ByVal User As Object, ByVal What As Download)
With DirectCast(User, UserData)
_NextWNM = .WaitNotificationMode
If _NextWNM = UserData.WNM.SkipTemp Or _NextWNM = UserData.WNM.SkipCurrent Then _NextWNM = UserData.WNM.Notify
LastDownloadDate.Value = Now
LastRequestsCount.Value = .RequestsCount
End With
End Sub
Friend Overrides Sub DownloadDone(ByVal What As Download)
_NextWNM = UserData.WNM.Notify
LastDownloadDate.Value = Now
ActiveJobs -= 1
If HashUpdateRequired Then MyMainLOG = "Check your Instagram credentials"
End Sub
#End Region
<PropertyUpdater(NameOf(Hash))>
Friend Function GatherInstaHash() As Boolean
Try
If Not Responser.Cookies.ListExists Then Throw New Exception("Instagram cookies does not set")
Dim rs As New RParams("=""([^""]+?ConsumerLibCommons[^""]+?.js)""", Nothing, 1) With {.MatchTimeOut = 10}
Dim r$ = Responser.GetResponse("https://instagram.com",, EDP.ThrowException)
If Not r.IsEmptyString Then
Dim hStr$ = RegexReplace(r, rs)
If Not hStr.IsEmptyString Then
Do While Left(hStr, 1) = "/" : hStr = Right(hStr, hStr.Length - 1) : Loop
hStr = $"https://instagram.com/{hStr}"
r = Responser.GetResponse(hStr,, EDP.ThrowException)
If Not r.IsEmptyString Then
rs = New RParams("generatePaginationActionCreators.+?.profilePosts.byUserId.get.+?queryId:.([\d\w\S]+?)""", Nothing, 1) With {.MatchTimeOut = 10}
Dim h$ = RegexReplace(r, rs)
If Not h.IsEmptyString Then
Hash.Value = h
HashUpdateRequired.Value = False
Return True
End If
End If
End If
End If
Return False
Catch ex As Exception
HashUpdateRequired.Value = True
Hash.Value = String.Empty
Return ErrorsDescriber.Execute(EDP.SendInLog + EDP.ReturnValue, ex, "[SiteSettings.GaterInstaHash]", False)
End Try
End Function
Friend Overrides Function GetSpecialDataF(ByVal URL As String) As IEnumerable(Of UserMedia)
Return UserData.GetVideoInfo(URL, Responser)
End Function
Friend Overrides Sub UserOptions(ByRef Options As Object, ByVal OpenForm As Boolean)
If Options Is Nothing OrElse Not TypeOf Options Is EditorExchangeOptions Then Options = New EditorExchangeOptions(Me)
If OpenForm Then
Using f As New OptionsForm(Options) : f.ShowDialog() : End Using
End If
End Sub
End Class
End Namespace

View File

@@ -7,56 +7,222 @@
' This program is distributed in the hope that it will be useful, ' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY ' but WITHOUT ANY WARRANTY
Imports PersonalUtilities.Functions.XML Imports PersonalUtilities.Functions.XML
Imports PersonalUtilities.Functions.Messaging
Imports PersonalUtilities.Functions.RegularExpressions
Imports PersonalUtilities.Tools.WEB
Imports PersonalUtilities.Tools.WebDocuments.JSON Imports PersonalUtilities.Tools.WebDocuments.JSON
Imports SCrawler.API.Base Imports SCrawler.API.Base
Imports System.Threading Imports System.Threading
Imports System.Net Imports System.Net
Imports UStates = SCrawler.API.Base.UserMedia.States
Imports UTypes = SCrawler.API.Base.UserMedia.Types Imports UTypes = SCrawler.API.Base.UserMedia.Types
Namespace API.Instagram Namespace API.Instagram
Friend Class UserData : Inherits UserDataBase Friend Class UserData : Inherits UserDataBase
Friend Overrides Property Site As Sites = Sites.Instagram Private Const MaxPostsCount As Integer = 200
''' <summary>Video downloader initializer</summary> Private Const Name_LastCursor As String = "LastCursor"
Private Sub New() Private Const Name_FirstLoadingDone As String = "FirstLoadingDone"
Private Const Name_GetStories As String = "GetStories"
Private Const Name_GetTagged As String = "GetTaggedData"
Private Const Name_TaggedChecked As String = "TaggedChecked"
Private ReadOnly Property MySiteSettings As SiteSettings
Get
Return DirectCast(HOST.Source, SiteSettings)
End Get
End Property
Private ReadOnly _SavedPostsIDs As New List(Of String)
Private LastCursor As String = String.Empty
Private FirstLoadingDone As Boolean = False
Friend Property GetStories As Boolean
Friend Property GetTaggedData As Boolean
Friend Overrides Function ExchangeOptionsGet() As Object
Return New EditorExchangeOptions(HOST.Source) With {.GetStories = GetStories, .GetTagged = GetTaggedData}
End Function
Friend Overrides Sub ExchangeOptionsSet(ByVal Obj As Object)
If Not Obj Is Nothing AndAlso TypeOf Obj Is EditorExchangeOptions Then
With DirectCast(Obj, EditorExchangeOptions)
GetStories = .GetStories
GetTaggedData = .GetTagged
End With
End If
End Sub End Sub
''' <summary>Default initializer</summary> Friend Sub New()
Friend Sub New(ByVal u As UserInfo, Optional ByVal _LoadUserInformation As Boolean = True)
User = u
If _LoadUserInformation Then LoadUserInformation()
End Sub End Sub
Protected Overrides Sub LoadUserInformation_OptionalFields(ByRef Container As XmlFile, ByVal Loading As Boolean)
If Loading Then
LastCursor = Container.Value(Name_LastCursor)
FirstLoadingDone = Container.Value(Name_FirstLoadingDone).FromXML(Of Boolean)(False)
GetStories = Container.Value(Name_GetStories).FromXML(Of Boolean)(CBool(MySiteSettings.GetStories.Value))
GetTaggedData = Container.Value(Name_GetTagged).FromXML(Of Boolean)(CBool(MySiteSettings.GetTagged.Value))
TaggedChecked = Container.Value(Name_TaggedChecked).FromXML(Of Boolean)(False)
Else
Container.Add(Name_LastCursor, LastCursor)
Container.Add(Name_FirstLoadingDone, FirstLoadingDone.BoolToInteger)
Container.Add(Name_GetStories, GetStories.BoolToInteger)
Container.Add(Name_GetTagged, GetTaggedData.BoolToInteger)
Container.Add(Name_TaggedChecked, TaggedChecked.BoolToInteger)
End If
End Sub
#Region "Download data"
Protected Overrides Sub DownloadDataF(ByVal Token As CancellationToken) Protected Overrides Sub DownloadDataF(ByVal Token As CancellationToken)
DownloadData(String.Empty, Token) Try
_InstaHash = String.Empty
HasError = False
If Not LastCursor.IsEmptyString Then
DownloadData(LastCursor, Sections.Timeline, Token)
ThrowAny(Token)
If Not HasError Then FirstLoadingDone = True
End If
If Not HasError Then
DownloadData(String.Empty, Sections.Timeline, Token)
ThrowAny(Token)
If Not HasError Then FirstLoadingDone = True
End If
If FirstLoadingDone Then LastCursor = String.Empty
If IsSavedPosts Then
DownloadPosts(Token)
ElseIf MySiteSettings.StoriesAndTaggedReady Then
If GetStories Then DownloadData(String.Empty, Sections.Stories, Token)
If GetTaggedData Then DownloadData(String.Empty, Sections.Tagged, Token)
End If
If WaitNotificationMode = WNM.SkipTemp Or WaitNotificationMode = WNM.SkipCurrent Then WaitNotificationMode = WNM.Notify
Catch eex As ExitException
Catch ex As Exception
ProcessException(ex, Token, "[API.Instagram.UserData.DownloadDataF", False)
End Try
End Sub End Sub
Private _InstaHash As String = String.Empty Private _InstaHash As String = String.Empty
Private Overloads Sub DownloadData(ByVal Cursor As String, ByVal Token As CancellationToken) Friend Enum Sections
Timeline
Tagged
Stories
End Enum
#Region "429 bypass"
Friend Property RequestsCount As Integer = 0
Friend Enum WNM As Integer
Notify = 0
SkipCurrent = 1
SkipAll = 2
SkipTemp = 3
End Enum
Friend WaitNotificationMode As WNM = WNM.Notify
Private Caught429 As Boolean = False
Private ProgressTempSet As Boolean = False
Private Const InstAborted As String = "InstAborted"
Private Function Ready() As Boolean
With MySiteSettings
If Not .ReadyForDownload Then
If WaitNotificationMode = WNM.Notify Then
Dim m As New MMessage("Instagram [too many requests] error." & vbCr &
$"The program suggests waiting {If(.LastApplyingValue, 0)} minutes." & vbCr &
"What do you want to do?", "Waiting for Instagram download...",
{
New MsgBoxButton("Wait") With {.ToolTip = "Wait and ask again when the error is found."},
New MsgBoxButton("Wait (disable current") With {.ToolTip = "Wait and skip future prompts while downloading the current profile."},
New MsgBoxButton("Abort") With {.ToolTip = "Abort operation"},
New MsgBoxButton("Wait (disable all)") With {.ToolTip = "Wait and skip future prompts while downloading the current session."}
},
vbExclamation) With {.ButtonsPerRow = 2, .DefaultButton = 0, .CancelButton = 2}
Select Case MsgBoxE(m).Index
Case 1 : WaitNotificationMode = WNM.SkipCurrent
Case 2 : Throw New OperationCanceledException("Instagram download operation aborted") With {.HelpLink = InstAborted}
Case 3 : WaitNotificationMode = WNM.SkipAll
Case Else : WaitNotificationMode = WNM.SkipTemp
End Select
End If
If Not ProgressTempSet Then Progress.InformationTemporary = $"Waiting until { .GetWaitDate().ToString(ParsersDataDateProvider)}"
ProgressTempSet = True
Return False
Else
Return True
End If
End With
End Function
Private Sub ReconfigureAwaiter()
If WaitNotificationMode = WNM.SkipTemp Then WaitNotificationMode = WNM.Notify
If Caught429 Then Caught429 = False
ProgressTempSet = False
End Sub
Private Sub NextRequest(ByVal StartWait As Boolean)
With MySiteSettings
If StartWait And RequestsCount > 0 And (RequestsCount Mod .RequestsWaitTimerTaskCount.Value) = 0 Then Thread.Sleep(CInt(.RequestsWaitTimer.Value))
If RequestsCount >= MaxPostsCount - 5 Then Thread.Sleep(CInt(.SleepTimerOnPostsLimit.Value))
End With
End Sub
#End Region
Private Const StoriesFolder As String = "Stories"
Private Const TaggedFolder As String = "Tagged"
Private TaggedChecked As Boolean = False
Private Overloads Sub DownloadData(ByVal Cursor As String, ByVal Section As Sections, ByVal Token As CancellationToken)
Dim URL$ = String.Empty Dim URL$ = String.Empty
Dim StoriesList As List(Of String) = Nothing
Dim StoriesRequested As Boolean = False
Dim _DownloadComplete As Boolean = False
LastCursor = Cursor
Try
Do While Not _DownloadComplete
ThrowAny(Token)
If Not Ready() Then Thread.Sleep(10000) : ThrowAny(Token) : Continue Do
ReconfigureAwaiter()
Try Try
Dim n As EContainer, nn As EContainer, node As EContainer Dim n As EContainer, nn As EContainer, node As EContainer
Dim HasNextPage As Boolean = False Dim HasNextPage As Boolean = False
Dim EndCursor$ = String.Empty Dim EndCursor$ = String.Empty
Dim PostID$ = String.Empty, PostDate$ = String.Empty Dim PostID$ = String.Empty, PostDate$ = String.Empty, SpecFolder$ = String.Empty
Dim TaggedCount%
Dim ENode() As Object = Nothing
NextRequest(True)
'Check environment 'Check environment
If Cursor.IsEmptyString And _InstaHash.IsEmptyString Then _InstaHash = Settings(Sites.Instagram).InstaHash If Cursor.IsEmptyString And _InstaHash.IsEmptyString Then _
If _InstaHash.IsEmptyString Then Throw New ArgumentNullException("InstHash", "Query hash is null") _InstaHash = CStr(If(IsSavedPosts, MySiteSettings.HashSavedPosts, MySiteSettings.Hash).Value)
AuthNullException.ThrowIfNull(Section, IsSavedPosts, MySiteSettings)
If ID.IsEmptyString Then GetUserId() If ID.IsEmptyString Then GetUserId()
If ID.IsEmptyString Then Throw New ArgumentException("User ID is not detected", "ID") If ID.IsEmptyString Then Throw New ArgumentException("User ID is not detected", "ID")
'Create query 'Create query
Dim vars$ = "{""id"":" & ID & ",""first"":12,""after"":""" & Cursor & """}" Select Case Section
Case Sections.Timeline
Dim vars$ = "{""id"":" & ID & ",""first"":50,""after"":""" & Cursor & """}"
vars = SymbolsConverter.ASCII.EncodeSymbolsOnly(vars) vars = SymbolsConverter.ASCII.EncodeSymbolsOnly(vars)
URL = $"https://www.instagram.com/graphql/query/?query_hash={_InstaHash}&variables={vars}" URL = $"https://www.instagram.com/graphql/query/?query_hash={_InstaHash}&variables={vars}"
ENode = {"data", "user", 0}
Case Sections.Tagged
URL = $"https://i.instagram.com/api/v1/usertags/{ID}/feed/?count=50&max_id={Cursor}"
ENode = {"items"}
SpecFolder = TaggedFolder
Case Sections.Stories
If Not StoriesRequested Then
StoriesList = GetStoriesList()
StoriesRequested = True
MySiteSettings.TooManyRequests(False)
RequestsCount += 1
ThrowAny(Token)
End If
If StoriesList.ListExists Then
GetStoriesData(StoriesList, Token)
MySiteSettings.TooManyRequests(False)
RequestsCount += 1
End If
If StoriesList.ListExists Then
Continue Do
Else
Throw New ExitException(_DownloadComplete)
End If
End Select
'Get response 'Get response
Dim r$ = Responser.GetResponse(URL,, EDP.ThrowException) Dim r$ = Responser.GetResponse(URL,, EDP.ThrowException)
Settings(Sites.Instagram).InstagramTooManyRequests(False) MySiteSettings.TooManyRequests(False)
RequestsCount += 1
ThrowAny(Token) ThrowAny(Token)
'Data 'Data
If Not r.IsEmptyString Then If Not r.IsEmptyString Then
Using j As EContainer = JsonDocument.Parse(r).XmlIfNothing Using j As EContainer = JsonDocument.Parse(r).XmlIfNothing
n = j.ItemF({"data", "user", 0}).XmlIfNothing n = j.ItemF(ENode).XmlIfNothing
If n.Count > 0 Then If n.Count > 0 Then
Select Case Section
Case Sections.Timeline
If n.Contains("page_info") Then If n.Contains("page_info") Then
With n("page_info") With n("page_info")
HasNextPage = .Value("has_next_page").FromXML(Of Boolean)(False) HasNextPage = .Value("has_next_page").FromXML(Of Boolean)(False)
@@ -68,45 +234,146 @@ Namespace API.Instagram
For Each nn In n For Each nn In n
ThrowAny(Token) ThrowAny(Token)
node = nn(0).XmlIfNothing node = nn(0).XmlIfNothing
If IsSavedPosts Then
PostID = node.Value("id") PostID = node.Value("shortcode")
If Not PostID.IsEmptyString And _TempPostsList.Contains(PostID) Then Exit Sub If Not PostID.IsEmptyString Then
_TempPostsList.Add(PostID) If _TempPostsList.Contains(PostID) Then Throw New ExitException(_DownloadComplete) Else _SavedPostsIDs.Add(PostID)
PostDate = node.Value("taken_at_timestamp")
ObtainMedia(node, PostID, PostDate)
Next
End If End If
Else
PostID = node.Value("id")
If Not PostID.IsEmptyString And _TempPostsList.Contains(PostID) Then Throw New ExitException(_DownloadComplete)
_TempPostsList.Add(PostID)
PostDate = node.Value("taken_at_timestamp")
If Not CheckDatesLimit(PostDate, DateProvider) Then Throw New ExitException(_DownloadComplete)
ObtainMedia(node, PostID, PostDate, SpecFolder)
End If
Next
End If
Case Sections.Tagged
HasNextPage = j.Value("more_available").FromXML(Of Boolean)(False)
EndCursor = j.Value("next_max_id")
For Each nn In n
PostID = $"Tagged_{nn.Value("id")}"
If Not PostID.IsEmptyString And _TempPostsList.Contains(PostID) Then Throw New ExitException(_DownloadComplete)
_TempPostsList.Add(PostID)
ObtainMedia2(nn, PostID, SpecFolder)
Next
If Not TaggedChecked Then
TaggedCount = j.Value("total_count").FromXML(Of Integer)(0)
TaggedChecked = True
If TaggedCount > 200 Then
Dim a% = MsgBoxE({$"The number of tagged posts is {TaggedCount.NumToString(New ANumbers With {
.FormatOptions = ANumbers.Options.GroupIntegral})}" & vbCr &
"The tagged data download operation can take a long time.", "Too much tagged data"}, vbExclamation,,,
{"Continue",
New MsgBoxButton("Disable and cancel") With {
.ToolTip = "Disable downloading tagged data and cancel downloading tagged data."},
"Cancel"})
If a > 0 Then
If a = 1 Then GetTaggedData = False
Throw New ExitException(_DownloadComplete)
End If
End If
End If
End Select
Else Else
If j.Value("status") = "ok" AndAlso j({"data", "user"}).XmlIfNothing.Count = 0 AndAlso _TempMediaList.Count = 0 Then If j.Value("status") = "ok" AndAlso j({"data", "user"}).XmlIfNothing.Count = 0 AndAlso _TempMediaList.Count = 0 Then
Settings(Sites.Instagram).InstaHashUpdateRequired.Value = True MySiteSettings.HashUpdateRequired.Value = True
UserExists = False UserExists = False
Exit Sub Throw New ExitException(_DownloadComplete)
End If End If
End If End If
End Using End Using
End If
If HasNextPage And Not EndCursor.IsEmptyString Then DownloadData(EndCursor, Token)
Catch oex As OperationCanceledException When Token.IsCancellationRequested
Catch dex As ObjectDisposedException When Disposed
Catch ex As Exception
If Responser.StatusCode = HttpStatusCode.NotFound Then
UserExists = False
ElseIf Responser.StatusCode = HttpStatusCode.BadRequest Then
MyMainLOG = "Instagram credentials have expired"
Settings(Sites.Instagram).InstaHashUpdateRequired.Value = True
ElseIf Responser.StatusCode = 429 Then
Settings(Sites.Instagram).InstagramTooManyRequests(True)
Else Else
Settings(Sites.Instagram).InstaHashUpdateRequired.Value = True Throw New ExitException(_DownloadComplete)
LogError(ex, $"data downloading error [{URL}]")
End If End If
HasError = True _DownloadComplete = True
Finally If HasNextPage And Not EndCursor.IsEmptyString Then DownloadData(EndCursor, Section, Token)
_InstaHash = String.Empty Catch iane As AuthNullException
ErrorsDescriber.Execute(EDP.SendInLog, iane)
Throw New ExitException(_DownloadComplete)
Catch eex As ExitException
Throw eex
Catch oex As OperationCanceledException When Token.IsCancellationRequested
Exit Do
Catch dex As ObjectDisposedException When Disposed
Exit Do
Catch ex As Exception
If DownloadingException(ex, $"data downloading error [{URL}]") = 1 Then Continue Do Else Exit Do
End Try
Loop
Catch eex2 As ExitException
If (Section = Sections.Timeline Or Section = Sections.Tagged) And Not Cursor.IsEmptyString Then Throw eex2
Catch oex2 As OperationCanceledException When Token.IsCancellationRequested Or oex2.HelpLink = InstAborted
If oex2.HelpLink = InstAborted Then HasError = True
Catch DoEx As Exception
ProcessException(DoEx, Token, $"data downloading error [{URL}]")
End Try End Try
End Sub End Sub
Private Sub ObtainMedia(ByVal node As EContainer, ByVal PostID As String, ByVal PostDate As String) Private Sub DownloadPosts(ByVal Token As CancellationToken)
Dim URL$ = String.Empty
Dim _DownloadComplete As Boolean = False
Dim _Index% = 0
Try
Do While Not _DownloadComplete
ThrowAny(Token)
If Not Ready() Then Thread.Sleep(10000) : ThrowAny(Token) : Continue Do
ReconfigureAwaiter()
Try
Dim r$
Dim j As EContainer, jj As EContainer
Dim _MediaObtained As Boolean
If _SavedPostsIDs.Count > 0 And _Index <= _SavedPostsIDs.Count - 1 Then
Dim e As New ErrorsDescriber(EDP.ThrowException)
For i% = _Index To _SavedPostsIDs.Count - 1
_Index = i
URL = $"https://instagram.com/p/{_SavedPostsIDs(i)}/?__a=1"
ThrowAny(Token)
NextRequest(((i + 1) Mod 5) = 0)
ThrowAny(Token)
r = Responser.GetResponse(URL,, e)
MySiteSettings.TooManyRequests(False)
RequestsCount += 1
If Not r.IsEmptyString Then
j = JsonDocument.Parse(r)
If Not j Is Nothing Then
_MediaObtained = False
If j.Contains({"graphql", "shortcode_media"}) Then
With j({"graphql", "shortcode_media"}).XmlIfNothing
If .Count > 0 Then ObtainMedia(.Self, _SavedPostsIDs(i), String.Empty, String.Empty) : _MediaObtained = True
End With
End If
If Not _MediaObtained AndAlso j.Contains("items") Then
With j("items")
If .Count > 0 Then
For Each jj In .Self : ObtainMedia2(jj, _SavedPostsIDs(i)) : Next
End If
End With
End If
j.Dispose()
End If
End If
Next
End If
_DownloadComplete = True
Catch oex As OperationCanceledException When Token.IsCancellationRequested
Exit Do
Catch dex As ObjectDisposedException When Disposed
Exit Do
Catch ex As Exception
If DownloadingException(ex, $"downloading saved posts error [{URL}]") = 1 Then Continue Do Else Exit Do
End Try
Loop
Catch oex2 As OperationCanceledException When Token.IsCancellationRequested Or oex2.HelpLink = InstAborted
If oex2.HelpLink = InstAborted Then HasError = True
Catch DoEx As Exception
ProcessException(DoEx, Token, $"downloading saved posts error [{URL}]")
End Try
End Sub
#End Region
#Region "Obtain Media"
Private Sub ObtainMedia(ByVal node As EContainer, ByVal PostID As String, ByVal PostDate As String, ByVal SpecFolder As String)
Dim CreateMedia As Action(Of EContainer) = Dim CreateMedia As Action(Of EContainer) =
Sub(ByVal e As EContainer) Sub(ByVal e As EContainer)
Dim t As UTypes = If(e.Value("is_video").FromXML(Of Boolean)(False), UTypes.Video, UTypes.Picture) Dim t As UTypes = If(e.Value("is_video").FromXML(Of Boolean)(False), UTypes.Video, UTypes.Picture)
@@ -116,7 +383,7 @@ Namespace API.Instagram
Else Else
tmpValue = e.Value("video_url") tmpValue = e.Value("video_url")
End If End If
If Not tmpValue.IsEmptyString Then _TempMediaList.ListAddValue(MediaFromData(t, tmpValue, PostID, PostDate), LNC) If Not tmpValue.IsEmptyString Then _TempMediaList.ListAddValue(MediaFromData(t, tmpValue, PostID, PostDate, SpecFolder), LNC)
End Sub End Sub
If node.Contains({"edge_sidecar_to_children", "edges"}) Then If node.Contains({"edge_sidecar_to_children", "edges"}) Then
For Each edge As EContainer In node({"edge_sidecar_to_children", "edges"}) : CreateMedia(edge("node").XmlIfNothing) : Next For Each edge As EContainer In node({"edge_sidecar_to_children", "edges"}) : CreateMedia(edge("node").XmlIfNothing) : Next
@@ -124,6 +391,89 @@ Namespace API.Instagram
CreateMedia(node) CreateMedia(node)
End If End If
End Sub End Sub
Private Sub ObtainMedia2(ByVal n As EContainer, ByVal PostID As String, Optional ByVal SpecialFolder As String = Nothing,
Optional ByVal DateObj As String = Nothing)
Try
Dim img As Predicate(Of EContainer) = Function(_img) Not _img.Name.IsEmptyString AndAlso _img.Name.StartsWith("image_versions") AndAlso _img.Count > 0
Dim vid As Predicate(Of EContainer) = Function(_vid) Not _vid.Name.IsEmptyString AndAlso _vid.Name.StartsWith("video_versions") AndAlso _vid.Count > 0
Dim ss As Func(Of EContainer, Sizes) = Function(_ss) New Sizes(_ss.Value("width"), _ss.Value("url"))
Dim mDate As Func(Of EContainer, String) = Function(ByVal elem As EContainer) As String
If elem.Contains("taken_at") Then
Return elem.Value("taken_at")
ElseIf elem.Contains("imported_taken_at") Then
Return elem.Value("imported_taken_at")
Else
Dim ev$ = elem.Value("device_timestamp")
If Not ev.IsEmptyString Then
If ev.Length > 10 Then
Return elem.Value("device_timestamp").Substring(0, 10)
Else
Return ev
End If
Else
Return String.Empty
End If
End If
End Function
If n.Count > 0 Then
Dim l As New List(Of Sizes)
Dim d As EContainer
Dim t%
'8 - gallery
'2 - one video
'1 - one picture
t = n.Value("media_type").FromXML(Of Integer)(-1)
If t >= 0 Then
Select Case t
Case 1
If n.Contains(img) Then
t = n.Value("media_type").FromXML(Of Integer)(-1)
DateObj = mDate(n)
If t >= 0 Then
With n.ItemF({img, "candidates"}).XmlIfNothing
If .Count > 0 Then
l.Clear()
l.ListAddList(.Select(ss), LNC)
If l.Count > 0 Then
l.Sort()
_TempMediaList.ListAddValue(MediaFromData(UTypes.Picture, l.First.Data, PostID, DateObj, SpecialFolder), LNC)
l.Clear()
End If
End If
End With
End If
End If
Case 2
If n.Contains(vid) Then
DateObj = mDate(n)
With n.ItemF({vid}).XmlIfNothing
If .Count > 0 Then
l.Clear()
l.ListAddList(.Select(ss), LNC)
If l.Count > 0 Then
l.Sort()
_TempMediaList.ListAddValue(MediaFromData(UTypes.Video, l.First.Data, PostID, DateObj, SpecialFolder), LNC)
l.Clear()
End If
End If
End With
End If
Case 8
DateObj = mDate(n)
With n("carousel_media").XmlIfNothing
If .Count > 0 Then
For Each d In .Self : ObtainMedia2(d, PostID, SpecialFolder, DateObj) : Next
End If
End With
End Select
End If
l.Clear()
End If
Catch ex As Exception
ErrorsDescriber.Execute(EDP.SendInLog, ex, "API.Instagram.ObtainMedia2")
End Try
End Sub
#End Region
Private Sub GetUserId() Private Sub GetUserId()
Try Try
Dim r$ = Responser.GetResponse($"https://www.instagram.com/{Name}/?__a=1",, EDP.ThrowException) Dim r$ = Responser.GetResponse($"https://www.instagram.com/{Name}/?__a=1",, EDP.ThrowException)
@@ -136,106 +486,126 @@ Namespace API.Instagram
If Responser.StatusCode = HttpStatusCode.NotFound Or Responser.StatusCode = HttpStatusCode.BadRequest Then If Responser.StatusCode = HttpStatusCode.NotFound Or Responser.StatusCode = HttpStatusCode.BadRequest Then
Throw ex Throw ex
Else Else
LogError(ex, "get instagram user id") LogError(ex, "get Instagram user id")
End If End If
End Try End Try
End Sub End Sub
#Region "Pinned stories"
Private Sub GetStoriesData(ByRef StoriesList As List(Of String), ByVal Token As CancellationToken)
Const ReqUrl$ = "https://i.instagram.com/api/v1/feed/reels_media/?{0}"
Dim tmpList As IEnumerable(Of String)
Dim qStr$, r$, sFolder$, storyID$
Dim i% = -1
Dim jj As EContainer, s As EContainer
ThrowAny(Token)
If StoriesList.ListExists Then
tmpList = StoriesList.Take(5)
If tmpList.ListExists Then
qStr = String.Format(ReqUrl, tmpList.Select(Function(q) $"reel_ids=highlight:{q}").ListToString(, "&"))
r = Responser.GetResponse(qStr,, EDP.ThrowException)
ThrowAny(Token)
If Not r.IsEmptyString Then
Using j As EContainer = JsonDocument.Parse(r).XmlIfNothing
If j.Contains("reels") Then
For Each jj In j("reels")
i += 1
sFolder = jj.Value("title")
storyID = jj.Value("id").Replace("highlight:", String.Empty)
If sFolder.IsEmptyString Then sFolder = $"Story_{storyID}"
If sFolder.IsEmptyString Then sFolder = $"Story_{i}"
sFolder = $"{StoriesFolder}\{sFolder}"
If Not storyID.IsEmptyString Then storyID &= ":"
With jj("items").XmlIfNothing
If .Count > 0 Then
For Each s In .Self : ThrowAny(Token) : ObtainMedia2(s, storyID & s.Value("id"), sFolder) : Next
End If
End With
Next
End If
End Using
End If
StoriesList.RemoveRange(0, tmpList.Count)
End If
End If
End Sub
Private Function GetStoriesList() As List(Of String)
Try
Dim r$ = Responser.GetResponse($"https://i.instagram.com/api/v1/highlights/{ID}/highlights_tray/",, EDP.ThrowException)
If Not r.IsEmptyString Then
Using j As EContainer = JsonDocument.Parse(r).XmlIfNothing()("tray").XmlIfNothing
If j.Count > 0 Then Return j.Select(Function(jj) jj.Value("id").Replace("highlight:", String.Empty)).ListIfNothing
End Using
End If
Return Nothing
Catch ex As Exception
DownloadingException(ex, "API.Instagram.GetStoriesList")
Return Nothing
End Try
End Function
#End Region
Protected Overrides Sub ReparseVideo(ByVal Token As CancellationToken) Protected Overrides Sub ReparseVideo(ByVal Token As CancellationToken)
End Sub End Sub
Protected Overrides Sub DownloadContent(ByVal Token As CancellationToken) Protected Overrides Sub DownloadContent(ByVal Token As CancellationToken)
Try DownloadContentDefault(Token)
Dim i%
Dim dCount% = 0, dTotal% = 0
ThrowAny(Token)
If _ContentNew.Count > 0 Then
_ContentNew.RemoveAll(Function(c) c.URL.IsEmptyString)
If _ContentNew.Count > 0 Then
MyFile.Exists(SFO.Path)
Dim MyDir$ = MyFile.CutPath.PathNoSeparator
Dim vsf As Boolean = SeparateVideoFolderF
Dim f As SFile
Dim v As UserMedia
Using w As New WebClient
If vsf Then SFileShares.SFileExists($"{MyDir}\Video\", SFO.Path)
MainProgress.TotalCount += _ContentNew.Count
For i = 0 To _ContentNew.Count - 1
ThrowAny(Token)
v = _ContentNew(i)
v.State = UStates.Tried
If v.File.IsEmptyString Then
f = v.URL
Else
f = v.File
End If
f.Separator = "\"
f.Path = MyDir
If v.URL_BASE.IsEmptyString Then v.URL_BASE = v.URL
If Not v.File.IsEmptyString AndAlso Not v.URL_BASE.IsEmptyString Then
Try
If v.Type = UTypes.Video And vsf Then f.Path = $"{f.PathWithSeparator}Video"
w.DownloadFile(v.URL_BASE, f.ToString)
Select Case v.Type
Case UTypes.Video : DownloadedVideos += 1 : _CountVideo += 1
Case UTypes.Picture : DownloadedPictures += 1 : _CountPictures += 1
End Select
v.File = ChangeFileNameByProvider(f, v)
v.State = UStates.Downloaded
Catch wex As Exception
ErrorDownloading(f, v.URL_BASE)
End Try
Else
v.State = UStates.Skipped
End If
_ContentNew(i) = v
If DownloadTopCount.HasValue AndAlso dCount >= DownloadTopCount.Value Then
MainProgress.Perform(_ContentNew.Count - dTotal)
Exit Sub
Else
dTotal += 1
MainProgress.Perform()
End If
Next
End Using
End If
End If
Catch oex As OperationCanceledException When Token.IsCancellationRequested
Catch dex As ObjectDisposedException When Disposed
Catch ex As Exception
LogError(ex, "content downloading error")
HasError = True
End Try
End Sub End Sub
Private Shared Function MediaFromData(ByVal t As UTypes, ByVal _URL As String, ByVal PostID As String, ByVal PostDate As String) As UserMedia ''' <summary>
''' <inheritdoc cref="UserDataBase.DownloadingException(Exception, String)"/><br/>
''' 1 - continue
''' </summary>
Protected Overrides Function DownloadingException(ByVal ex As Exception, ByVal Message As String, Optional ByVal FromPE As Boolean = False) As Integer
If Responser.StatusCode = HttpStatusCode.NotFound Then
UserExists = False
ElseIf Responser.StatusCode = HttpStatusCode.BadRequest Then
HasError = True
MyMainLOG = "Instagram credentials have expired"
MySiteSettings.HashUpdateRequired.Value = True
ElseIf Responser.StatusCode = 429 Then
With MySiteSettings
Dim WaiterExists As Boolean = .LastApplyingValue.HasValue
.TooManyRequests(True)
If Not WaiterExists Then .LastApplyingValue = 2
End With
Caught429 = True
MyMainLOG = $"Number of requests before error 429: {RequestsCount}"
Return 1
Else
MySiteSettings.HashUpdateRequired.Value = True
If Not FromPE Then LogError(ex, Message) : HasError = True
Return 0
End If
Return 2
End Function
Private Shared Function MediaFromData(ByVal t As UTypes, ByVal _URL As String, ByVal PostID As String, ByVal PostDate As String,
Optional ByVal SpecialFolder As String = Nothing) As UserMedia
_URL = LinkFormatterSecure(RegexReplace(_URL.Replace("\", String.Empty), LinkPattern)) _URL = LinkFormatterSecure(RegexReplace(_URL.Replace("\", String.Empty), LinkPattern))
Dim m As New UserMedia(_URL, t) With {.Post = New UserPost With {.ID = PostID}} Dim m As New UserMedia(_URL, t) With {.Post = New UserPost With {.ID = PostID}}
If Not m.URL.IsEmptyString Then m.File = CStr(RegexReplace(m.URL, FilesPattern)) If Not m.URL.IsEmptyString Then m.File = CStr(RegexReplace(m.URL, FilesPattern))
If Not PostDate.IsEmptyString Then m.Post.Date = AConvert(Of Date)(PostDate, Declarations.DateProvider, Nothing) Else m.Post.Date = Nothing If Not PostDate.IsEmptyString Then m.Post.Date = AConvert(Of Date)(PostDate, DateProvider, Nothing) Else m.Post.Date = Nothing
m.SpecialFolder = SpecialFolder
Return m Return m
End Function End Function
Friend Shared Function GetVideoInfo(ByVal URL As String) As IEnumerable(Of UserMedia) Friend Shared Function GetVideoInfo(ByVal URL As String, ByVal r As Response) As IEnumerable(Of UserMedia)
Try Try
If Not URL.IsEmptyString AndAlso URL.Contains("instagram.com") Then If Not URL.IsEmptyString AndAlso URL.Contains("instagram.com") Then
Do While Right(URL, 1) = "/" : URL = Left(URL, URL.Length - 1) : Loop Dim PID$ = RegexReplace(URL, RParams.DMS(".*?instagram.com/p/([_\w\d]+)", 1))
URL = $"{URL}/?__a=1" If Not PID.IsEmptyString Then
Using t As New UserData Using t As New UserData
t.Responser = New PersonalUtilities.Tools.WEB.Response t.Responser = New Response
t.Responser.Copy(Settings(Sites.Instagram).Responser) t.Responser.Copy(r)
Dim r$ = t.Responser.GetResponse(URL,, EDP.ThrowException) t._SavedPostsIDs.Add(PID)
If Not r.IsEmptyString Then t.DownloadPosts(Nothing)
Using j As EContainer = JsonDocument.Parse(r).XmlIfNothing Return ListAddList(Nothing, t._TempMediaList)
Dim node As EContainer = j({"graphql", "shortcode_media"}).XmlIfNothing
If node.Count > 0 Then t.ObtainMedia(node, String.Empty, String.Empty)
End Using End Using
End If End If
If t._TempMediaList.Count > 0 Then Return ListAddList(Nothing, t._TempMediaList)
End Using
End If End If
Return Nothing Return Nothing
Catch ex As Exception Catch ex As Exception
Return ErrorsDescriber.Execute(EDP.ShowMainMsg + EDP.SendInLog, ex, "Instagram standalone downloader: fetch media error") Return ErrorsDescriber.Execute(EDP.ShowMainMsg + EDP.SendInLog, ex, "Instagram standalone downloader: fetch media error")
End Try End Try
End Function End Function
Protected Overrides Sub Dispose(ByVal disposing As Boolean)
If Not disposedValue And disposing Then _SavedPostsIDs.Clear()
MyBase.Dispose(disposing)
End Sub
End Class End Class
End Namespace End Namespace

View File

@@ -10,262 +10,27 @@ Imports PersonalUtilities.Tools
Imports PersonalUtilities.Forms.Toolbars Imports PersonalUtilities.Forms.Toolbars
Imports PersonalUtilities.Functions.XML Imports PersonalUtilities.Functions.XML
Imports SCrawler.API.Base Imports SCrawler.API.Base
Imports SCrawler.Plugin.Hosts
Imports System.Threading Imports System.Threading
Imports SCrawler.API.Reddit.RedditViewExchange
Imports View = SCrawler.API.Reddit.IRedditView.View
Imports Period = SCrawler.API.Reddit.IRedditView.Period
Namespace API.Reddit Namespace API.Reddit
Friend Class Channel : Implements ICollection(Of UserPost), IEquatable(Of Channel), IComparable(Of Channel), Friend Class Channel : Implements ICollection(Of UserPost), IEquatable(Of Channel), IComparable(Of Channel),
IRangeSwitcherContainer(Of UserPost), ILoaderSaver, IMyEnumerator(Of UserPost), IChannelLimits, IUserData, IDisposable IRangeSwitcherContainer(Of UserPost), ILoaderSaver, IMyEnumerator(Of UserPost), IChannelLimits, IRedditView, IDisposable
#Region "XML Nodes' Names" #Region "XML Nodes' Names"
Private Const Name_Name As String = "Name" Private Const Name_Name As String = "Name"
Private Const Name_ID As String = "ID" Private Const Name_ID As String = "ID"
Private Const Name_Date As String = "Date" Private Const Name_Date As String = "Date"
Private Const Name_PostsNode As String = "Posts" Private Const Name_PostsNode As String = "Posts"
Private Const Name_UsersAdded As String = "UsersAdded"
Private Const Name_UsersExistent As String = "UsersExistent"
Private Const Name_PostsDownloaded As String = "PostsDownloaded"
#End Region #End Region
Friend Const DefaultDownloadLimitCount As Integer = 1000 Friend Const DefaultDownloadLimitCount As Integer = 1000
#Region "IUserData Support" Friend ReadOnly Property Site As String = RedditSite
Private Event OnUserUpdated As IUserData.OnUserUpdatedEventHandler Implements IUserData.OnUserUpdated Friend Property Name As String
Friend Property Instance As IUserData Friend Property ID As String
Private Property IUserData_ParseUserMediaOnly As Boolean = False Implements IUserData.ParseUserMediaOnly
Private Property IUserData_Exists As Boolean Implements IUserData.Exists
Get
Return Instance.Exists
End Get
Set(ByVal e As Boolean)
End Set
End Property
Private Property IUserData_Suspended As Boolean Implements IUserData.Suspended
Get
Return Instance.Suspended
End Get
Set(ByVal s As Boolean)
End Set
End Property
Private ReadOnly Property IUserData_IsCollection As Boolean Implements IUserData.IsCollection
Get
Return Instance.IsCollection
End Get
End Property
Private Property IUserData_CollectionName As String Implements IUserData.CollectionName
Get
Return Instance.CollectionName
End Get
Set(ByVal NewName As String)
Instance.CollectionName = NewName
End Set
End Property
Private ReadOnly Property IUserData_IncludedInCollection As Boolean Implements IUserData.IncludedInCollection
Get
Return Instance.IncludedInCollection
End Get
End Property
Private ReadOnly Property IUserData_Labels As List(Of String) Implements IUserData.Labels
Get
Return Instance.Labels
End Get
End Property
Private ReadOnly Property IUserData_IsChannel As Boolean = True Implements IUserData.IsChannel
Private Property IUserData_ReadyForDownload As Boolean Implements IUserData.ReadyForDownload
Get
Return Instance.ReadyForDownload
End Get
Set(ByVal IsReady As Boolean)
Instance.ReadyForDownload = IsReady
End Set
End Property
Private Property IUserData_File As SFile Implements IUserData.File
Get
Return Instance.File
End Get
Set(ByVal NewFile As SFile)
Instance.File = NewFile
End Set
End Property
Private Property IUserData_FileExists As Boolean Implements IUserData.FileExists
Get
Return Instance.FileExists
End Get
Set(ByVal IsExists As Boolean)
Instance.FileExists = IsExists
End Set
End Property
Private Property IUserData_DownloadedPictures As Integer Implements IUserData.DownloadedPictures
Get
Return Instance.DownloadedPictures
End Get
Set(ByVal c As Integer)
Instance.DownloadedPictures = c
End Set
End Property
Private Property IUserData_DownloadedVideos As Integer Implements IUserData.DownloadedVideos
Get
Return Instance.DownloadedVideos
End Get
Set(ByVal c As Integer)
Instance.DownloadedVideos = c
End Set
End Property
Private ReadOnly Property IUserData_DownloadedTotal(Optional Total As Boolean = True) As Integer Implements IUserData.DownloadedTotal
Get
Return Instance.DownloadedTotal
End Get
End Property
Private ReadOnly Property IUserData_DownloadedInformation As String Implements IUserData.DownloadedInformation
Get
Return Instance.DownloadedInformation
End Get
End Property
Private Property IUserData_HasError As Boolean Implements IUserData.HasError
Get
Return Instance.HasError
End Get
Set(ByVal e As Boolean)
Instance.HasError = e
End Set
End Property
Private ReadOnly Property IUserData_FitToAddParams As Boolean Implements IUserData.FitToAddParams
Get
Return Instance.FitToAddParams
End Get
End Property
Private ReadOnly Property IUserData_LVIKey As String Implements IUserData.LVIKey
Get
Return Instance.LVIKey
End Get
End Property
Private ReadOnly Property IUserData_LVIIndex As Integer Implements IUserData.LVIIndex
Get
Return Instance.LVIIndex
End Get
End Property
Private Property IUserData_DownloadImages As Boolean Implements IUserData.DownloadImages
Get
Return Instance.DownloadImages
End Get
Set(ByVal d As Boolean)
Instance.DownloadImages = d
End Set
End Property
Private Property IUserData_DownloadVideos As Boolean Implements IUserData.DownloadVideos
Get
Return Instance.DownloadVideos
End Get
Set(ByVal d As Boolean)
Instance.DownloadVideos = d
End Set
End Property
Private ReadOnly Property IUserData_Self As IUserData Implements IUserData.Self
Get
Return Instance
End Get
End Property
Private Property IUserData_DownloadTopCount As Integer? Implements IUserData.DownloadTopCount
Get
Return Instance.DownloadTopCount
End Get
Set(ByVal c As Integer?)
Instance.DownloadTopCount = c
End Set
End Property
Friend Property Site As Sites = Sites.Reddit Implements IContentProvider.Site
Private Property IUserData_FriendlyName As String Implements IContentProvider.FriendlyName
Get
Return Instance.FriendlyName
End Get
Set(ByVal NewName As String)
Instance.FriendlyName = NewName
End Set
End Property
Private Property IUserData_Description As String Implements IContentProvider.Description
Get
Return Instance.Description
End Get
Set(ByVal d As String)
Instance.Description = d
End Set
End Property
Private Property IUserData_Favorite As Boolean Implements IContentProvider.Favorite
Get
Return Instance.Favorite
End Get
Set(ByVal f As Boolean)
Instance.Favorite = f
End Set
End Property
Private Property IUserData_Temporary As Boolean Implements IContentProvider.Temporary
Get
Return Instance.Temporary
End Get
Set(ByVal t As Boolean)
Instance.Temporary = t
End Set
End Property
Private Sub IUserData_SetPicture(ByVal f As SFile) Implements IUserData.SetPicture
Instance.SetPicture(f)
End Sub
Private Sub IUserData_LoadUserInformation() Implements IUserData.LoadUserInformation
Instance.LoadUserInformation()
End Sub
Private Sub IUserData_UpdateUserInformation() Implements IUserData.UpdateUserInformation
Instance.UpdateUserInformation()
End Sub
Private Sub IUserData_OpenFolder() Implements IUserData.OpenFolder
Instance.OpenFolder()
End Sub
Private Sub IUserData_OpenSite() Implements IContentProvider.OpenSite
Instance.OpenSite()
End Sub
Private Sub IUserData_DownloadData(ByVal Token As CancellationToken) Implements IContentProvider.DownloadData
DownloadData(Token, False, Nothing)
End Sub
Private Function IUserData_GetPicture() As Image Implements IUserData.GetPicture
Return Instance.GetPicture()
End Function
Private Function IUserData_GetLVI(ByVal Destination As ListView) As ListViewItem Implements IUserData.GetLVI
Return Instance.GetLVI(Destination)
End Function
Private Function IUserData_GetLVIGroup(ByVal Destination As ListView) As ListViewGroup Implements IUserData.GetLVIGroup
Return Instance.GetLVIGroup(Destination)
End Function
Private Function IUserData_Delete() As Integer Implements IUserData.Delete
Return DirectCast(Instance, UserDataBase).DeleteF(Me)
End Function
Private Function IUserData_MoveFiles(ByVal CollectionName As String) As Boolean Implements IUserData.MoveFiles
Return DirectCast(Instance, UserDataBase).MoveFilesF(Me, CollectionName)
End Function
#End Region
Private _Name As String = String.Empty
Friend Property Name As String Implements IUserData.Name
Get
If IsRegularChannel Then
Return Instance.Name
Else
Return _Name
End If
End Get
Set(ByVal NewName As String)
If IsRegularChannel Then
Instance.Name = NewName
Else
_Name = NewName
End If
End Set
End Property
Private _ID As String = String.Empty
Friend Property ID As String Implements IUserData.ID
Get
If IsRegularChannel Then
Return Instance.ID
Else
Return _ID
End If
End Get
Set(ByVal NewID As String)
If IsRegularChannel Then
Instance.ID = NewID
Else
_ID = NewID
End If
End Set
End Property
Friend ReadOnly Property CUser As UserInfo Friend ReadOnly Property CUser As UserInfo
Get Get
Return New UserInfo(Me) Return New UserInfo(Me)
@@ -273,6 +38,7 @@ Namespace API.Reddit
End Property End Property
Friend ReadOnly Property PostsLatest As List(Of UserPost) Friend ReadOnly Property PostsLatest As List(Of UserPost)
Friend ReadOnly Property Posts As List(Of UserPost) Friend ReadOnly Property Posts As List(Of UserPost)
Friend ReadOnly Property PostsNames As List(Of String)
Friend ReadOnly Property PostsAll As List(Of UserPost) Friend ReadOnly Property PostsAll As List(Of UserPost)
Get Get
Return ListAddList(Nothing, Posts).ListAddList(PostsLatest).ListSort Return ListAddList(Nothing, Posts).ListAddList(PostsLatest).ListSort
@@ -295,6 +61,14 @@ Namespace API.Reddit
Return $"{ChannelsCollection.ChannelsPath.PathWithSeparator}{ID}.xml" Return $"{ChannelsCollection.ChannelsPath.PathWithSeparator}{ID}.xml"
End Get End Get
End Property End Property
Private ReadOnly Property FilePosts As SFile
Get
Dim f As SFile = File
f.Name &= "_Posts"
f.Extension = "txt"
Return f
End Get
End Property
Friend ReadOnly Property CachePath As SFile Friend ReadOnly Property CachePath As SFile
Get Get
Return $"{ChannelsCollection.ChannelsPathCache.PathWithSeparator}{ID}\" Return $"{ChannelsCollection.ChannelsPathCache.PathWithSeparator}{ID}\"
@@ -310,11 +84,62 @@ Namespace API.Reddit
Return Posts(Index) Return Posts(Index)
End Get End Get
End Property End Property
Private ReadOnly Property Range As RangeSwitcher(Of UserPost) Friend Property ViewMode As View = View.New Implements IRedditView.ViewMode
Friend Property ViewPeriod As Period = Period.All Implements IRedditView.ViewPeriod
Friend Sub SetView(ByVal Options As IRedditView) Implements IRedditView.SetView
If Not Options Is Nothing Then
ViewMode = Options.ViewMode
ViewPeriod = Options.ViewPeriod
End If
End Sub
#Region "Statistics support"
Private ReadOnly CountOfAddedUsers As List(Of Integer)
Private ReadOnly CountOfLoadedPostsPerSession As List(Of Integer)
Friend ReadOnly Property ChannelExistentUserNames As List(Of String)
Private _FirstUserAdded As Boolean = False
Friend Sub UserAdded(ByVal UserName As String, Optional ByVal IsAdded As Boolean = True)
If Not _FirstUserAdded Then CountOfAddedUsers.Add(0) : _FirstUserAdded = True
Dim v% = CountOfAddedUsers.Last
v += IIf(IsAdded, 1, -1)
If v < 0 Then v = 0
CountOfAddedUsers(CountOfAddedUsers.Count - 1) = v
If Not ChannelExistentUserNames.Contains(UserName) Then ChannelExistentUserNames.Add(UserName)
End Sub
Friend Sub UpdateUsersStats()
If Posts.Count > 0 Or PostsLatest.Count > 0 Then
ChannelExistentUserNames.ListAddList((From p As UserPost In PostsAll
Where Not p.UserID.IsEmptyString AndAlso
Settings.UsersList.Exists(Function(u) u.Site = Site And u.Name = p.UserID)
Select p.UserID), LAP.NotContainsOnly)
ChannelExistentUserNames.RemoveAll(Function(u) Not Settings.UsersList.Exists(Function(uu) uu.Site = Site And uu.Name = u))
End If
End Sub
Friend Function GetChannelStats(ByVal Extended As Boolean) As String
UpdateUsersStats()
Dim s$ = String.Empty
Dim p As New ANumbers With {.FormatOptions = ANumbers.Options.GroupIntegral}
If Extended Then
s.StringAppendLine($"Users added from this channel: {CountOfAddedUsers.Sum.NumToString(p)}")
s.StringAppendLine($"Users added from this channel (avg): {CountOfAddedUsers.DefaultIfEmpty(0).Average.RoundDown.NumToString(p)}")
s.StringAppendLine($"Users added from this channel (session): {CountOfAddedUsers.LastOrDefault.NumToString(p)}")
s.StringAppendLine($"Posts downloaded (avg): {CountOfLoadedPostsPerSession.DefaultIfEmpty(0).Average.RoundUp.NumToString(p)}")
s.StringAppendLine($"Posts downloaded (session): {CountOfLoadedPostsPerSession.LastOrDefault.NumToString(p)}")
s.StringAppendLine($"My users in this channel: {ChannelExistentUserNames.Count.NumToString(p)}")
Else
s.StringAppend($"Users: {CountOfAddedUsers.Sum.NumToString(p)} (avg: {CountOfAddedUsers.DefaultIfEmpty(0).Average.RoundDown.NumToString(p)}; s: {CountOfAddedUsers.LastOrDefault.NumToString(p)})")
s.StringAppend($"Posts: {CountOfLoadedPostsPerSession.DefaultIfEmpty(0).Average.RoundUp.NumToString(p)} (s: {CountOfLoadedPostsPerSession.LastOrDefault.NumToString(p)})", "; ")
s.StringAppend($"My users: {ChannelExistentUserNames.Count.NumToString(p)}", "; ")
End If
Return s
End Function
#End Region
#Region "Limits Support" #Region "Limits Support"
Private _DownloadLimitCount As Integer? = Nothing Private _DownloadLimitCount As Integer? = Nothing
Friend Property DownloadLimitCount As Integer? Implements IChannelLimits.DownloadLimitCount Friend Property DownloadLimitCount As Integer? Implements IChannelLimits.DownloadLimitCount
Get Get
If Not ViewMode = View.New And AutoGetLimits Then
Return _DownloadLimitCount
Else
If AutoGetLimits Then If AutoGetLimits Then
If LatestParsedDate.HasValue OrElse Not DownloadLimitPost.IsEmptyString Then If LatestParsedDate.HasValue OrElse Not DownloadLimitPost.IsEmptyString Then
Return Nothing Return Nothing
@@ -326,6 +151,7 @@ Namespace API.Reddit
Else Else
Return _DownloadLimitCount Return _DownloadLimitCount
End If End If
End If
End Get End Get
Set(ByVal NewLimit As Integer?) Set(ByVal NewLimit As Integer?)
_DownloadLimitCount = NewLimit _DownloadLimitCount = NewLimit
@@ -334,12 +160,16 @@ Namespace API.Reddit
Private _DownloadLimitPost As String = String.Empty Private _DownloadLimitPost As String = String.Empty
Friend Property DownloadLimitPost As String Implements IChannelLimits.DownloadLimitPost Friend Property DownloadLimitPost As String Implements IChannelLimits.DownloadLimitPost
Get Get
If Not ViewMode = View.New And AutoGetLimits Then
Return _DownloadLimitPost
Else
Dim PID$ = ListAddList(Nothing, Posts, LAP.NotContainsOnly).ListAddList(PostsLatest, LAP.NotContainsOnly).ListSort.FirstOrDefault.ID Dim PID$ = ListAddList(Nothing, Posts, LAP.NotContainsOnly).ListAddList(PostsLatest, LAP.NotContainsOnly).ListSort.FirstOrDefault.ID
If AutoGetLimits And Not PID.IsEmptyString Then If AutoGetLimits And Not PID.IsEmptyString Then
Return PID Return PID
Else Else
Return _DownloadLimitPost Return _DownloadLimitPost
End If End If
End If
End Get End Get
Set(ByVal NewLimit As String) Set(ByVal NewLimit As String)
_DownloadLimitPost = NewLimit _DownloadLimitPost = NewLimit
@@ -348,11 +178,15 @@ Namespace API.Reddit
Private _DownloadLimitDate As Date? = Nothing Private _DownloadLimitDate As Date? = Nothing
Friend Property DownloadLimitDate As Date? Implements IChannelLimits.DownloadLimitDate Friend Property DownloadLimitDate As Date? Implements IChannelLimits.DownloadLimitDate
Get Get
If Not ViewMode = View.New And AutoGetLimits Then
Return _DownloadLimitDate
Else
If AutoGetLimits And LatestParsedDate.HasValue Then If AutoGetLimits And LatestParsedDate.HasValue Then
Return LatestParsedDate Return LatestParsedDate
Else Else
Return _DownloadLimitDate Return _DownloadLimitDate
End If End If
End If
End Get End Get
Set(ByVal NewLimit As Date?) Set(ByVal NewLimit As Date?)
_DownloadLimitDate = NewLimit _DownloadLimitDate = NewLimit
@@ -371,32 +205,31 @@ Namespace API.Reddit
DownloadLimitDate = .DownloadLimitDate DownloadLimitDate = .DownloadLimitDate
AutoGetLimits = .AutoGetLimits AutoGetLimits = .AutoGetLimits
End With End With
If Not ViewMode = View.New And AutoGetLimits Then
DownloadLimitDate = Nothing
DownloadLimitCount = Nothing
DownloadLimitPost = String.Empty
End If
End Sub End Sub
Friend Property AutoGetLimits As Boolean = True Implements IChannelLimits.AutoGetLimits Friend Property AutoGetLimits As Boolean = True Implements IChannelLimits.AutoGetLimits
#End Region #End Region
Friend ReadOnly IsRegularChannel As Boolean = False Friend ReadOnly Property HOST As SettingsHost
Friend Sub New() Friend Sub New()
Posts = New List(Of UserPost) Posts = New List(Of UserPost)
PostsLatest = New List(Of UserPost) PostsLatest = New List(Of UserPost)
Range = New RangeSwitcher(Of UserPost)(Me) PostsNames = New List(Of String)
CountOfAddedUsers = New List(Of Integer)
CountOfLoadedPostsPerSession = New List(Of Integer)
ChannelExistentUserNames = New List(Of String)
HOST = Settings(RedditSiteKey)
End Sub End Sub
Friend Sub New(ByVal f As SFile) Friend Sub New(ByVal f As SFile)
Me.New Me.New
LoadData(f, False) LoadData(f, False)
End Sub End Sub
Friend Sub New(ByVal u As UserInfo, Optional ByVal _LoadUserInformation As Boolean = True)
Me.New
Instance = New UserData(u, _LoadUserInformation) With {.SaveToCache = False, .SkipExistsUsers = False, .ChannelInfo = Me}
AutoGetLimits = True
DirectCast(Instance, UserData).SetLimit(Me)
IsRegularChannel = True
End Sub
Public Shared Widening Operator CType(ByVal f As SFile) As Channel Public Shared Widening Operator CType(ByVal f As SFile) As Channel
Return New Channel(f) Return New Channel(f)
End Operator End Operator
Public Shared Widening Operator CType(ByVal c As Channel) As UserDataBase
Return DirectCast(c.Instance, UserDataBase)
End Operator
Public Overrides Function ToString() As String Public Overrides Function ToString() As String
If Not Name.IsEmptyString Then If Not Name.IsEmptyString Then
Return Name Return Name
@@ -405,28 +238,33 @@ Namespace API.Reddit
End If End If
End Function End Function
Friend Sub Delete() Friend Sub Delete()
If File.Exists Then File.Delete() File.Delete(, SFODelete.DeleteToRecycleBin)
FilePosts.Delete(, SFODelete.DeleteToRecycleBin)
End Sub End Sub
Friend Sub DownloadData(ByVal Token As CancellationToken, Optional ByVal SkipExists As Boolean = True, Friend Sub DownloadData(ByVal Token As CancellationToken, Optional ByVal SkipExists As Boolean = True,
Optional ByVal p As MyProgress = Nothing) Optional ByVal p As MyProgress = Nothing)
Try Try
_Downloading = True _Downloading = True
If Not Instance Is Nothing Then Using d As New UserData With {
Instance.DownloadData(Token)
Else
Using d As New UserData(CUser, False, False) With {
.Progress = p, .Progress = p,
.SaveToCache = True, .SaveToCache = True,
.SkipExistsUsers = SkipExists, .SkipExistsUsers = SkipExists,
.ChannelInfo = Me .ChannelInfo = Me
} }
d.SetLimit(Me) With d
d.DownloadData(Token) .SetEnvironment(HOST, CUser, False)
.RemoveUpdateHandlers()
.SetLimit(Me)
.SetView(Me)
.DownloadData(Token)
End With
Dim b% = Posts.Count
Posts.ListAddList(d.GetNewChannelPosts(), LAP.NotContainsOnly) Posts.ListAddList(d.GetNewChannelPosts(), LAP.NotContainsOnly)
If Posts.Count - b > 0 Then CountOfLoadedPostsPerSession.Add(Posts.Count - b)
Posts.Sort() Posts.Sort()
LatestParsedDate = If(Posts.FirstOrDefault(Function(pp) pp.Date.HasValue).Date, LatestParsedDate) LatestParsedDate = If(Posts.FirstOrDefault(Function(pp) pp.Date.HasValue).Date, LatestParsedDate)
UpdateUsersStats()
End Using End Using
End If
Catch oex As OperationCanceledException When Token.IsCancellationRequested Catch oex As OperationCanceledException When Token.IsCancellationRequested
Finally Finally
_Downloading = False _Downloading = False
@@ -467,21 +305,12 @@ Namespace API.Reddit
Friend Overloads Function Equals(ByVal Other As Channel) As Boolean Implements IEquatable(Of Channel).Equals Friend Overloads Function Equals(ByVal Other As Channel) As Boolean Implements IEquatable(Of Channel).Equals
Return ID = Other.ID Return ID = Other.ID
End Function End Function
Private Overloads Function Equals(ByVal Other As UserDataBase) As Boolean Implements IEquatable(Of UserDataBase).Equals
If Not Instance Is Nothing Then
Return Instance.Equals(Other)
Else
Return False
End If
End Function
Public Overloads Overrides Function Equals(ByVal Obj As Object) As Boolean Public Overloads Overrides Function Equals(ByVal Obj As Object) As Boolean
If Not Obj Is Nothing Then If Not Obj Is Nothing Then
If TypeOf Obj Is String Then If TypeOf Obj Is String Then
Return ID = CStr(Obj) Return ID = CStr(Obj)
ElseIf TypeOf Obj Is Channel Then ElseIf TypeOf Obj Is Channel Then
Return Equals(DirectCast(Obj, Channel)) Return Equals(DirectCast(Obj, Channel))
ElseIf TypeOf Obj Is UserDataBase Then
Return Equals(DirectCast(Obj, UserDataBase))
End If End If
End If End If
Return False Return False
@@ -495,22 +324,6 @@ Namespace API.Reddit
Return ID.CompareTo(Other.ID) Return ID.CompareTo(Other.ID)
End If End If
End Function End Function
Private Overloads Function CompareTo(ByVal Other As UserDataBase) As Integer Implements IComparable(Of UserDataBase).CompareTo
If Not Instance Is Nothing Then
Return Instance.CompareTo(Other)
Else
Return 0
End If
End Function
Private Overloads Function CompareTo(ByVal Obj As Object) As Integer Implements IComparable.CompareTo
If TypeOf Obj Is Channel Then
Return CompareTo(DirectCast(Obj, Channel))
ElseIf TypeOf Obj Is UserDataBase And Not Instance Is Nothing Then
Return Instance.CompareTo(Obj)
Else
Return 0
End If
End Function
#End Region #End Region
#Region "ILoaderSaver Support" #Region "ILoaderSaver Support"
Friend Overloads Function LoadData(Optional ByVal f As SFile = Nothing, Optional ByVal e As ErrorsDescriber = Nothing) As Boolean Implements ILoaderSaver.Load Friend Overloads Function LoadData(Optional ByVal f As SFile = Nothing, Optional ByVal e As ErrorsDescriber = Nothing) As Boolean Implements ILoaderSaver.Load
@@ -522,9 +335,16 @@ Namespace API.Reddit
x.LoadData() x.LoadData()
If x.Count > 0 Then If x.Count > 0 Then
Dim XMLDateProvider As New ADateTime(ADateTime.Formats.BaseDateTime) Dim XMLDateProvider As New ADateTime(ADateTime.Formats.BaseDateTime)
Dim lc As New ListAddParams(LAP.ClearBeforeAdd)
Name = x.Value(Name_Name) Name = x.Value(Name_Name)
ID = x.Value(Name_ID) ID = x.Value(Name_ID)
ViewMode = x.Value(Name_ViewMode).FromXML(Of Integer)(CInt(View.[New]))
ViewPeriod = x.Value(Name_ViewPeriod).FromXML(Of Integer)(CInt(Period.All))
If FilePosts.Exists Then PostsNames.ListAddList(FilePosts.GetText.StringToList(Of String)("|"), LNC)
LatestParsedDate = AConvert(Of Date)(x.Value(Name_Date), XMLDateProvider, Nothing) LatestParsedDate = AConvert(Of Date)(x.Value(Name_Date), XMLDateProvider, Nothing)
CountOfAddedUsers.ListAddList(x.Value(Name_UsersAdded).StringToList(Of Integer)("|"), lc)
CountOfLoadedPostsPerSession.ListAddList(x.Value(Name_PostsDownloaded).StringToList(Of Integer)("|"), lc)
ChannelExistentUserNames.ListAddList(x.Value(Name_UsersExistent).StringToList(Of String)("|"), LNC)
If Not PartialLoad Then If Not PartialLoad Then
With x(Name_PostsNode).XmlIfNothing With x(Name_PostsNode).XmlIfNothing
If .Count > 0 Then .ForEach(Sub(ee) PostsLatest.Add(New UserPost With { If .Count > 0 Then .ForEach(Sub(ee) PostsLatest.Add(New UserPost With {
@@ -539,9 +359,21 @@ Namespace API.Reddit
End Function End Function
Friend Overloads Function Save(Optional ByVal f As SFile = Nothing, Optional ByVal e As ErrorsDescriber = Nothing) As Boolean Implements ILoaderSaver.Save Friend Overloads Function Save(Optional ByVal f As SFile = Nothing, Optional ByVal e As ErrorsDescriber = Nothing) As Boolean Implements ILoaderSaver.Save
Dim XMLDateProvider As New ADateTime(ADateTime.Formats.BaseDateTime) Dim XMLDateProvider As New ADateTime(ADateTime.Formats.BaseDateTime)
UpdateUsersStats()
If Not ViewMode = View.New Then
Dim l As New List(Of String)
If Posts.Count > 0 Or PostsLatest.Count > 0 Then l.ListAddList((From p In PostsAll Where Not p.ID.IsEmptyString Select p.ID), LAP.NotContainsOnly)
l.ListAddList(PostsNames, LAP.NotContainsOnly)
If l.Count > 0 Then TextSaver.SaveTextToFile(l.ListToString(, "|"), FilePosts, True,, EDP.SendInLog)
End If
Using x As New XmlFile With {.AllowSameNames = True, .Name = "Channel"} Using x As New XmlFile With {.AllowSameNames = True, .Name = "Channel"}
x.Add(Name_Name, Name) x.Add(Name_Name, Name)
x.Add(Name_ID, ID) x.Add(Name_ID, ID)
x.Add(Name_ViewMode, CInt(ViewMode))
x.Add(Name_ViewPeriod, CInt(ViewPeriod))
x.Add(Name_UsersAdded, CountOfAddedUsers.ListToString(, "|"))
x.Add(Name_PostsDownloaded, CountOfLoadedPostsPerSession.ListToString(, "|"))
x.Add(Name_UsersExistent, ChannelExistentUserNames.ListToString(, "|"))
If Posts.Count > 0 Or PostsLatest.Count > 0 Then If Posts.Count > 0 Or PostsLatest.Count > 0 Then
Dim tmpPostList As List(Of UserPost) = Nothing Dim tmpPostList As List(Of UserPost) = Nothing
tmpPostList.ListAddList(Posts).ListAddList(PostsLatest) tmpPostList.ListAddList(Posts).ListAddList(PostsLatest)
@@ -568,19 +400,16 @@ Namespace API.Reddit
#End Region #End Region
#Region "IDisposable Support" #Region "IDisposable Support"
Private disposedValue As Boolean = False Private disposedValue As Boolean = False
Friend ReadOnly Property Disposed As Boolean Implements IUserData.Disposed
Get
Return disposedValue
End Get
End Property
Protected Overridable Overloads Sub Dispose(ByVal disposing As Boolean) Protected Overridable Overloads Sub Dispose(ByVal disposing As Boolean)
If Not disposedValue Then If Not disposedValue Then
If disposing Then If disposing Then
Posts.Clear() Posts.Clear()
PostsLatest.Clear() PostsLatest.Clear()
Range.Dispose() PostsNames.Clear()
If Not Instance Is Nothing Then Instance.Dispose() CountOfAddedUsers.Clear()
If CachePath.Exists(SFO.Path, False) Then CachePath.Delete(SFO.Path, False, False, EDP.SendInLog) CountOfLoadedPostsPerSession.Clear()
ChannelExistentUserNames.Clear()
CachePath.Delete(SFO.Path, SFODelete.None, EDP.SendInLog)
End If End If
disposedValue = True disposedValue = True
End If End If

View File

@@ -15,15 +15,39 @@ Namespace API.Reddit
Friend Shared ReadOnly Property ChannelsPath As SFile = $"{SettingsFolderName}\Channels\" Friend Shared ReadOnly Property ChannelsPath As SFile = $"{SettingsFolderName}\Channels\"
Friend Shared ReadOnly Property ChannelsPathCache As SFile = $"{Settings.GlobalPath.Value.PathWithSeparator}_CachedData\" Friend Shared ReadOnly Property ChannelsPathCache As SFile = $"{Settings.GlobalPath.Value.PathWithSeparator}_CachedData\"
Private ReadOnly Channels As List(Of Channel) Private ReadOnly Channels As List(Of Channel)
Friend Structure ChannelImage : Implements IEquatable(Of ChannelImage)
Friend File As SFile
Friend Channel As String
Friend Sub New(ByVal ChannelName As String, ByVal f As SFile)
Channel = ChannelName
File = f
End Sub
Friend Overloads Function Equals(ByVal Other As ChannelImage) As Boolean Implements IEquatable(Of ChannelImage).Equals
Return Channel = Other.Channel And File.File = Other.File.File
End Function
Public Overloads Overrides Function Equals(ByVal Obj As Object) As Boolean
Return Equals(DirectCast(Obj, ChannelImage))
End Function
End Structure
Friend ReadOnly Property Downloading As Boolean Friend ReadOnly Property Downloading As Boolean
Get Get
If Count > 0 Then Return Count > 0 AndAlso Channels.Exists(Function(c) c.Downloading)
Return Channels.Exists(Function(c) c.Downloading)
Else
Return False
End If
End Get End Get
End Property End Property
Friend Function GetUserFiles(ByVal UserName As String) As IEnumerable(Of ChannelImage)
Try
If Settings.ChannelsAddUserImagesFromAllChannels.Value And Count > 0 Then
Return Channels.SelectMany(Function(c) From p As UserPost In c.Posts Where p.UserID = UserName Select New ChannelImage(c.Name, p.CachedFile))
Else
Return Nothing
End If
Catch ex As Exception
Return ErrorsDescriber.Execute(EDP.SendInLog + EDP.ReturnValue, ex)
End Try
End Function
Friend Sub UpdateUsersStats()
If Channels.Count > 0 Then Channels.ForEach(Sub(c) c.UpdateUsersStats())
End Sub
#Region "Limits Support" #Region "Limits Support"
Friend Property DownloadLimitCount As Integer? Implements IChannelLimits.DownloadLimitCount Friend Property DownloadLimitCount As Integer? Implements IChannelLimits.DownloadLimitCount
<Obsolete("This property cannot be used in collections", True)> Private Property DownloadLimitPost As String Implements IChannelLimits.DownloadLimitPost <Obsolete("This property cannot be used in collections", True)> Private Property DownloadLimitPost As String Implements IChannelLimits.DownloadLimitPost
@@ -74,7 +98,6 @@ Namespace API.Reddit
Next Next
End If End If
Throw New ArgumentException($"Channel ID [{ChannelID}] does not found in channels collection", "ChannelID") With {.HelpLink = 1} Throw New ArgumentException($"Channel ID [{ChannelID}] does not found in channels collection", "ChannelID") With {.HelpLink = 1}
'Return Nothing
End Get End Get
End Property End Property
Friend Sub DownloadData(ByVal Token As CancellationToken, Optional ByVal SkipExists As Boolean = True, Friend Sub DownloadData(ByVal Token As CancellationToken, Optional ByVal SkipExists As Boolean = True,
@@ -111,7 +134,7 @@ Namespace API.Reddit
Return Count > 0 AndAlso Channels.Contains(_Item) Return Count > 0 AndAlso Channels.Contains(_Item)
End Function End Function
Friend Function Remove(ByVal _Item As Channel) As Boolean Implements ICollection(Of Channel).Remove Friend Function Remove(ByVal _Item As Channel) As Boolean Implements ICollection(Of Channel).Remove
If Count > 0 Then If Count > 0 And Not _Item Is Nothing Then
Dim i% = Channels.IndexOf(_Item) Dim i% = Channels.IndexOf(_Item)
If i >= 0 Then If i >= 0 Then
With Channels(i) : .Delete() : .Dispose() : End With With Channels(i) : .Delete() : .Dispose() : End With

Some files were not shown because too many files have changed in this diff Show More