2023.1.27.0

Plugins: added 'Interaction' to 'Provider' attribute; added 'IPropertyProvider' interface
Hosts: update classes to work with new options
Instagram: fixed pinned post reload
Twitter: advanced options for GIFs
UserCreatorForm: change icon based on the selected site
UserSearchForm: change search function

CONTRIBUTING.md

@@ -8,14 +8,16 @@ I welcome requests! Follow these steps to contribute:
 1. If you have a code change suggestion, you can post a replacement code block. I also accept pull requests.
 
 # How to build from source
-1. Delete the "PersonalUtilities" project from the solution.
+1. Delete the ```PersonalUtilities``` project from the solution.
-1. Delete the "PersonalUtilities.Notifications" project from the solution.
+1. Delete the ```PersonalUtilities.Notifications``` project from the solution.
+1. Delete the ```cURL``` folder from the solution.
+1. Delete the ```ffmpeg.exe``` from the solution.
 1. The following libraries must be added to project references with the '**Copy to output folder**' option:
-  - ```PersonalUtilities.dll```
+    - ```PersonalUtilities.dll```
-  - ```PersonalUtilities.Notifications.dll```
+    - ```PersonalUtilities.Notifications.dll```
-  - ```Microsoft.Toolkit.Uwp.Notifications.dll```
+    - ```Microsoft.Toolkit.Uwp.Notifications.dll```
-  - ```System.ValueTuple.dll```
+    - ```System.ValueTuple.dll```
-1. Import PersonalUtilities.Functions for the whole project.
+1. Import ```PersonalUtilities.Functions``` for the whole project.
 
 **Always use the correct libraries. You must download libraries from the same release date as the code commit date.**
 

Changelog.md

@@ -1,3 +1,82 @@
+# 2023.1.27.0
+
+*2023-01-27*
+
+- Added
+  - Advanced Twitter options for GIFs
+  - Changing the icon of the user creation form based on the selected site
+- Fixed
+  - Pinned Instagram posts reload every time
+- Plugins
+  - Added
+    - `Interaction` option to the `Provider` attribute
+	- `IPropertyProvider` interface
+
+# 2023.1.24.1
+
+*2023-01-24*
+
+- Added
+  - Icon for standalone downloader
+- Fixed
+  - (Issue #100) some Imgur albums won't download
+
+# 2023.1.24.0
+
+*2023-01-24*
+
+- Fixed
+  - (Issue #100) Imgur albums not downloading
+  - When deleting a collection with the 'ban' option, users in the collection are not banned
+
+# 2023.1.2.0
+
+*2023-01-02*
+
+- Added
+  - RedGifs: an ability to customize token refresh interval
+  - RedGifs: token refresh interval changed from 24 hours to 12 hours
+  - Updated labels collection
+- Fixed
+  - PornHub: bug in the downloader
+  - PornHub: download additional non-user videos
+  - Reddit: bug in standalone downloader
+  - Fixed a bug in the user list loading algorithm
+  - Notifications: pressing any button opens SCrawler
+
+# 2022.12.27.0
+
+*2022-12-27*
+
+- Added
+  - XVideos: added downloading 'Quickies'
+  - Instagram: added more enable/disable options
+- Fixed
+  - XVideos not downloading (sorry, I broke it in a previous release)
+
+# 2022.12.26.0
+
+*2022-12-26*
+
+**ATTENTION!**
+
+**Instagram requirements changed. Headers and cookies are now required to download Timeline, Stories and Saved posts; hash to download tagged posts. Please update your credentials.**
+
+**Instagram tagged posts no longer provide the total amount of tagged posts. I've corrected the tagged posts notification, but now I can't tell how many requests will be spent on downloading tagged posts. And from now on, one request will be spent on downloading each tagged post, because Instagram doesn't provide complete information about the tagged post with the site's response. In this case, if the number of tagged posts is 1000, 1000 requests will be spent. Be careful when downloading them. I highly recommend that you forcefully disable the downloading of tagged posts for a while.**
+
+- Added
+  - Updated user loading algorithm
+  - Channels button to tray context menu
+  - (Request #96) Add FFmpeg to x86 version
+- Fixed
+  - PornHub wrong behavior when downloading images
+  - Unable open XVideos user profile
+  - Cannot delete multiple collections at once
+  - Can't focus user from the download info form
+  - Instagram downloader not working
+  - (Issue #69) **RedGifs data is not downloading**. Again.
+  - Minor bugs
+
 # 2022.11.16.0
 
 *2022-11-16*

FAQ.md

@@ -42,7 +42,7 @@ A: How to request a new site you can read [here](CONTRIBUTING.md#how-to-request-
 
 #### Q: **Twitter/Instagram download failed.**
 
-A: Check your credentials. Both of these sites require cookies. Check your [Twitter tokens](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-find-twitter-tokens) and [Instagram settings](https://github.com/AAndyProgram/SCrawler/wiki/Settings#instagram-settings). If all settings are set, but nothing works, go to [create a new issue](https://github.com/AAndyProgram/SCrawler/issues). Don't forget to attach the LOG.
+A: Check your credentials. Both of these sites require cookies. Check your [Twitter tokens](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-find-twitter-tokens) and [Instagram settings](https://github.com/AAndyProgram/SCrawler/wiki/Settings#instagram). If all settings are set, but nothing works, go to [create a new issue](https://github.com/AAndyProgram/SCrawler/issues). Don't forget to attach the LOG.
 
 **[SITES REQUIREMENTS](https://github.com/AAndyProgram/SCrawler/wiki/Settings#sites-requirements)**
 

HowToSupport.md

@@ -6,6 +6,7 @@ You can support the program by:
   - :repeat: make a post about my program on your profile (Reddit, Twitter, Instagram and any other social networks)
   - :speech_balloon: tell your friends about the program
   - :heart: like the program on this site: https://alternativeto.net/software/scrawler/about/
+  - :heart: like the program on this site: https://www.softpedia.com/get/Internet/Download-Managers/Social-networks-crawler.shtml
   - suggest my program as an alternative ([on this site](https://alternativeto.net/software/scrawler/about/)) to any program you have used before
 
 I would be very grateful for any support! :blush:

README.md

@@ -2,6 +2,7 @@
 
 [![GitHub release (latest by date)](https://img.shields.io/github/v/release/AAndyProgram/SCrawler)](https://github.com/AAndyProgram/SCrawler/releases/latest)
 [![GitHub license](https://img.shields.io/github/license/AAndyProgram/SCrawler)](https://github.com/AAndyProgram/SCrawler/blob/main/LICENSE)
+[![GitHub all releases](https://img.shields.io/github/downloads/aandyprogram/scrawler/total?label=Total%20downloads)](https://github.com/AAndyProgram/SCrawler/releases)
 [![FAQ](https://img.shields.io/badge/FAQ-green)](FAQ.md)
 [![GUIDE](https://img.shields.io/badge/GUIDE-green)](https://github.com/AAndyProgram/SCrawler/wiki)
 [![How to support](https://img.shields.io/badge/HowToSupport-green)](HowToSupport.md)
@@ -36,18 +37,18 @@ Do you like this program? Consider adding to my coffee fund by making a donation
 - Download [saved Reddit, Twitter and Instagram posts](https://github.com/AAndyProgram/SCrawler/wiki/Home#saved-posts)
 - Add users from parsed channel
 - **Advanced user management**
-- **Automation** (downloading data automatically every ```X``` minutes)
+- **Automation** ([downloading data automatically](https://github.com/AAndyProgram/SCrawler/wiki/Settings#automation) every ```X``` minutes)
-- **Feed** (feed of downloaded media files)
+- **Feed** ([feed](https://github.com/AAndyProgram/SCrawler/wiki#feed) of downloaded media files)
 - Labeling users
-- Create download groups
+- Create [download groups](https://github.com/AAndyProgram/SCrawler/wiki/Settings#download-groups)
 - Adding users to favorites and temporary
-- Filter exists users by label or group
+- [Filter exists users](https://github.com/AAndyProgram/SCrawler/wiki#view) by label or group
 - Selection of media types you want to download (images only, videos only, both)
-- Download a special video, image or gallery
+- [Download a special video](https://github.com/AAndyProgram/SCrawler/wiki#download-separate-video), image or gallery
-- Making collections (grouping users into collections)
+- Making [collections](https://github.com/AAndyProgram/SCrawler/wiki#collections) (grouping users into collections)
 - Specifying a user folder (for downloading data to another location)
 - Changing user icons
-- Changing view modes
+- Changing [view modes](https://github.com/AAndyProgram/SCrawler/wiki#view)
 - ...and many others...
 
 # Supported sites
@@ -73,11 +74,11 @@ First, the program downloads the full profile. After the program downloads only
 
 ## Reddit
 
-The program parses all user posts, obtain MD5  images hash and compares them with existing ones to remove duplicates. Then the media will be downloaded.
+The program parses user posts, obtain MD5  images hash and compares them with existing ones to remove duplicates. Then the media will be downloaded.
 
 ## Other sites
 
-The program parses all user posts and compares file names with existing ones to remove duplicates. Then the media will be downloaded.
+The program parses user posts and compares file names with existing ones to remove duplicates. Then the media will be downloaded.
 
 ## How to request a new site
 
@@ -119,7 +120,7 @@ Read [here](CONTRIBUTING.md#how-to-request-a-new-site) about
 
 # Installation
 
-**Just download the [latest release](https://github.com/AAndyProgram/SCrawler/releases/latest), unzip the program archive to any folder, copy the file ```ffmpeg.exe``` into it and enjoy.** :blush:
+**Just download the [latest release](https://github.com/AAndyProgram/SCrawler/releases/latest), unzip the program archive to any folder and enjoy.** :blush:
 
 **Don't put program in the ```Program Files``` system folder (this is portable program and program settings are stored in the program folder)**
 
@@ -137,7 +138,7 @@ Read about how to make plugin [here](https://github.com/AAndyProgram/SCrawler/wi
 
 # How to support
 
-Read more about how to support the program [here](HowToSupport.md).
+Read about how to support the program [here](HowToSupport.md).
 
 # Settings and usage
 

SCrawler.PluginProvider/Attributes/Attributes.vb

@@ -100,6 +100,8 @@ Namespace Plugin.Attributes
         ''' <see langword="False"/> - only for conversion
         ''' </summary>
         Public FieldsChecker As Boolean = False
+        ''' <summary>Interaction with changing text field. Default: <see langword="False"/></summary>
+        Public Interaction As Boolean = False
         ''' <summary>Initialize a new Provider attribute. <see cref="IFormatProvider"/> is only allowed</summary>
         ''' <param name="PropertyName">The name of the property for which this provider is used</param>
         Public Sub New(ByVal PropertyName As String)

SCrawler.PluginProvider/Interfaces/IPropertyProvider.vb

@@ -0,0 +1,13 @@
+' Copyright (C) 2023  Andy https://github.com/AAndyProgram
+' This program is free software: you can redistribute it and/or modify
+' it under the terms of the GNU General Public License as published by
+' the Free Software Foundation, either version 3 of the License, or
+' (at your option) any later version.
+'
+' This program is distributed in the hope that it will be useful,
+' but WITHOUT ANY WARRANTY
+Namespace Plugin
+    Public Interface IPropertyProvider : Inherits IFormatProvider
+        Property PropertyName As String
+    End Interface
+End Namespace

SCrawler.PluginProvider/My Project/AssemblyInfo.vb

@@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices
 ' by using the '*' as shown below:
 ' <Assembly: AssemblyVersion("1.0.*")>
 
-<Assembly: AssemblyVersion("2022.11.16.0")>
+<Assembly: AssemblyVersion("2023.1.27.0")>
-<Assembly: AssemblyFileVersion("2022.11.16.0")>
+<Assembly: AssemblyFileVersion("2023.1.27.0")>
 <Assembly: NeutralResourcesLanguage("en")>

SCrawler.PluginProvider/Objects/PluginUserMedia.vb

@@ -34,6 +34,7 @@ Namespace Plugin
         Public Property PostDate As Date? Implements IUserMedia.PostDate
         Public Property SpecialFolder As String Implements IUserMedia.SpecialFolder
         Public Property Attempts As Integer Implements IUserMedia.Attempts
+        Public Property [Object] As Object Implements IUserMedia.Object
     End Structure
     Public Interface IUserMedia
         Property ContentType As Integer
@@ -46,5 +47,6 @@ Namespace Plugin
         Property PostDate As Date?
         Property SpecialFolder As String
         Property Attempts As Integer
+        Property [Object] As Object
     End Interface
 End Namespace

SCrawler.PluginProvider/SCrawler.PluginProvider.vbproj

@@ -102,6 +102,7 @@
   </ItemGroup>
   <ItemGroup>
     <Compile Include="Attributes\Attributes.vb" />
+    <Compile Include="Interfaces\IPropertyProvider.vb" />
     <Compile Include="Objects\ExchangeOptions.vb" />
     <Compile Include="ObjectInterfaces\ILogProvider.vb" />
     <Compile Include="Interfaces\IPluginContentProvider.vb" />

SCrawler/API/Base/M3U8Base.vb

@@ -28,7 +28,7 @@ Namespace API.Base
                 Return $"{Appender.StringTrimEnd("/")}/{File}"
             End If
         End Function
-        Friend Shared Function Download(ByVal URLs As List(Of String), ByVal DestinationFile As SFile, Optional ByVal Responser As Response = Nothing) As SFile
+        Friend Shared Function Download(ByVal URLs As List(Of String), ByVal DestinationFile As SFile, Optional ByVal Responser As Responser = Nothing) As SFile
             Dim CachePath As SFile = Nothing
             Try
                 If URLs.ListExists Then

SCrawler/API/Base/SiteSettingsBase.vb

@@ -17,8 +17,8 @@ Namespace API.Base
         Friend Overridable ReadOnly Property Icon As Icon Implements ISiteSettings.Icon
         Friend Overridable ReadOnly Property Image As Image Implements ISiteSettings.Image
         Private Property Logger As ILogProvider = LogConnector Implements ISiteSettings.Logger
-        Friend Overridable ReadOnly Property Responser As Response
+        Friend Overridable ReadOnly Property Responser As Responser
-        Private Property IResponserContainer_Responser As Response Implements IResponserContainer.Responser
+        Private Property IResponserContainer_Responser As Responser Implements IResponserContainer.Responser
             Get
                 Return Responser
             End Get
@@ -30,7 +30,7 @@ Namespace API.Base
         End Sub
         Friend Sub New(ByVal SiteName As String, ByVal CookiesDomain As String)
             Site = SiteName
-            Responser = New Response($"{SettingsFolderName}\Responser_{Site}.xml")
+            Responser = New Responser($"{SettingsFolderName}\Responser_{Site}.xml")
             With Responser
                 If .File.Exists Then
                     If EncryptCookies.CookiesEncrypted Then .CookiesEncryptKey = SettingsCLS.CookieEncryptKey

SCrawler/API/Base/Structures.vb

@@ -104,7 +104,7 @@ Namespace API.Base
                     Return Post.ID
                 End Get
                 Set(ByVal PostID As String)
-                    Post.ID = PostID
+                    Post = New UserPost(PostID, Post.Date)
                 End Set
             End Property
             Private Property IUserMedia_PostDate As Date? Implements IUserMedia.PostDate
@@ -112,7 +112,7 @@ Namespace API.Base
                     Return Post.Date
                 End Get
                 Set(ByVal PostDate As Date?)
-                    Post.Date = PostDate
+                    Post = New UserPost(Post.ID, PostDate)
                 End Set
             End Property
             Private Property IUserMedia_SpecialFolder As String Implements IUserMedia.SpecialFolder
@@ -131,6 +131,14 @@ Namespace API.Base
                     Me.Attempts = Attempts
                 End Set
             End Property
+            Private Property IUserMedia_Object As Object Implements IUserMedia.Object
+                Get
+                    Return Me.Object
+                End Get
+                Set(ByVal Obj As Object)
+                    Me.Object = Obj
+                End Set
+            End Property
 #End Region
             Friend Sub New(ByVal URL As String)
                 Me.URL = URL
@@ -142,7 +150,7 @@ Namespace API.Base
                 Me.New(URL)
                 Me.Type = Type
             End Sub
-            Friend Sub New(ByVal m As Plugin.IUserMedia)
+            Friend Sub New(ByVal m As IUserMedia)
                 [Type] = m.ContentType
                 URL = m.URL
                 URL_BASE = m.URL_BASE
@@ -152,6 +160,7 @@ Namespace API.Base
                 State = m.DownloadState
                 SpecialFolder = m.SpecialFolder
                 Attempts = m.Attempts
+                Me.Object = m.Object
             End Sub
             Friend Sub New(ByVal e As EContainer, ByVal UserInstance As IUserData)
                 Type = e.Attribute(Name_MediaType).Value.FromXML(Of Integer)(CInt(Types.Undefined))

SCrawler/API/Base/UserDataBase.vb

@@ -682,6 +682,7 @@ BlockNullPicture:
                         LastUpdated = AConvert(Of Date)(x.Value(Name_LastUpdated), ADateTime.Formats.BaseDateTime, Nothing)
                         ScriptUse = x.Value(Name_ScriptUse).FromXML(Of Boolean)(False)
                         ScriptData = x.Value(Name_ScriptData)
+                        'TODELETE: UserDataBase remove old 'merge' constant
 #Disable Warning BC40000
                         If x.Contains(Name_DataMerging) Then
                             DataMerging = x.Value(Name_DataMerging).FromXML(Of Boolean)(False)
@@ -843,7 +844,7 @@ BlockNullPicture:
         End Function
 #End Region
 #Region "Download functions and options"
-        Protected Responser As Response
+        Protected Responser As Responser
         Protected UseResponserClient As Boolean = False
         Friend Overridable Sub DownloadData(ByVal Token As CancellationToken) Implements IContentProvider.DownloadData
             Dim Canceled As Boolean = False
@@ -852,7 +853,7 @@ BlockNullPicture:
                 UpdateDataFiles()
                 UserDescriptionReset()
                 If Not Responser Is Nothing Then Responser.Dispose()
-                Responser = New Response
+                Responser = New Responser
                 If Not HOST.Responser Is Nothing Then Responser.Copy(HOST.Responser)
                 'TODO: UserDataBase remove [Responser.DecodersError]
                 Responser.DecodersError = New ErrorsDescriber(EDP.SendInLog + EDP.ReturnValue) With {
@@ -1022,12 +1023,14 @@ BlockNullPicture:
                                         End If
 
                                         If Not v.SpecialFolder.IsEmptyString Then
-                                            f.Path = $"{f.PathWithSeparator}{v.SpecialFolder}\".CSFileP.Path
+                                            f.Path = $"{f.PathWithSeparator}{v.SpecialFolder.StringTrimEnd("*")}\".CSFileP.Path
                                             f.Exists(SFO.Path)
                                         End If
                                         If __isVideo And vsf Then
-                                            f.Path = $"{f.PathWithSeparator}Video"
+                                            If v.SpecialFolder.IsEmptyString OrElse Not v.SpecialFolder.EndsWith("*") Then
-                                            If Not v.SpecialFolder.IsEmptyString Then f.Exists(SFO.Path)
+                                                f.Path = $"{f.PathWithSeparator}Video"
+                                                If Not v.SpecialFolder.IsEmptyString Then f.Exists(SFO.Path)
+                                            End If
                                         End If
 
                                         If v.Type = UTypes.m3u8 And UseInternalM3U8Function Then

SCrawler/API/Imgur/Envir.vb

@@ -15,7 +15,7 @@ Imports PersonalUtilities.Tools.Web.Documents.JSON
 Namespace API.Imgur
     Namespace Declarations
         Friend Module Imgur_Declarations
-            Friend ReadOnly PostRegex As RParams = RParams.DMS("/([\w\d]+?)(|\.[\w]{0,4})\Z", 1)
+            Friend ReadOnly PostRegex As RParams = RParams.DMS("/([^/]+?)(|#.*?|\.[\w]{0,4})(|\?.*?)\Z", 1)
         End Module
     End Namespace
     Friend NotInheritable Class Envir
@@ -70,11 +70,12 @@ Namespace API.Imgur
         Friend Shared Function GetVideoInfo(ByVal URL As String, Optional ByVal e As ErrorsDescriber = Nothing) As IEnumerable(Of UserMedia)
             Try
                 If Not URL.IsEmptyString AndAlso URL.ToLower.Contains("imgur") AndAlso Not Settings.ImgurClientID.IsEmptyString Then
-                    Dim img$ = GetImage(URL, EDP.ReturnValue)
+                    Dim imgList As List(Of String) = GetGallery(URL, EDP.ReturnValue)
-                    If Not img.IsEmptyString Then
+                    If imgList.ListExists Then
-                        Return {New UserMedia(img)}
+                        Return imgList.Select(Function(u) New UserMedia(u))
                     Else
-                        Return GetGallery(URL, EDP.ReturnValue).ListIfNothing.Select(Function(u) New UserMedia(u))
+                        Dim img$ = GetImage(URL, EDP.ReturnValue)
+                        If Not img.IsEmptyString Then Return {New UserMedia(img)}
                     End If
                 End If
                 Return Nothing

SCrawler/API/Instagram/AdditionalSettingsForm.Designer.vb

@@ -1,135 +0,0 @@
-' Copyright (C) 2023  Andy https://github.com/AAndyProgram
-' This program is free software: you can redistribute it and/or modify
-' it under the terms of the GNU General Public License as published by
-' the Free Software Foundation, either version 3 of the License, or
-' (at your option) any later version.
-'
-' This program is distributed in the hope that it will be useful,
-' but WITHOUT ANY WARRANTY
-Namespace API.Instagram
-    <Global.Microsoft.VisualBasic.CompilerServices.DesignerGenerated()>
-    Partial Friend Class AdditionalSettingsForm : Inherits System.Windows.Forms.Form
-        <System.Diagnostics.DebuggerNonUserCode()>
-        Protected Overrides Sub Dispose(ByVal disposing As Boolean)
-            Try
-                If disposing AndAlso components IsNot Nothing Then
-                    components.Dispose()
-                End If
-            Finally
-                MyBase.Dispose(disposing)
-            End Try
-        End Sub
-        Private components As System.ComponentModel.IContainer
-        <System.Diagnostics.DebuggerStepThrough()>
-        Private Sub InitializeComponent()
-            Dim CONTAINER_MAIN As System.Windows.Forms.ToolStripContainer
-            Dim TP_MAIN As System.Windows.Forms.TableLayoutPanel
-            Me.CH_DOWN_TIME = New System.Windows.Forms.CheckBox()
-            Me.CH_DOWN_TAG = New System.Windows.Forms.CheckBox()
-            Me.CH_DOWN_SAVED = New System.Windows.Forms.CheckBox()
-            CONTAINER_MAIN = New System.Windows.Forms.ToolStripContainer()
-            TP_MAIN = New System.Windows.Forms.TableLayoutPanel()
-            CONTAINER_MAIN.ContentPanel.SuspendLayout()
-            CONTAINER_MAIN.SuspendLayout()
-            TP_MAIN.SuspendLayout()
-            Me.SuspendLayout()
-            '
-            'CONTAINER_MAIN
-            '
-            '
-            'CONTAINER_MAIN.ContentPanel
-            '
-            CONTAINER_MAIN.ContentPanel.Controls.Add(TP_MAIN)
-            CONTAINER_MAIN.ContentPanel.Size = New System.Drawing.Size(234, 78)
-            CONTAINER_MAIN.Dock = System.Windows.Forms.DockStyle.Fill
-            CONTAINER_MAIN.LeftToolStripPanelVisible = False
-            CONTAINER_MAIN.Location = New System.Drawing.Point(0, 0)
-            CONTAINER_MAIN.Name = "CONTAINER_MAIN"
-            CONTAINER_MAIN.RightToolStripPanelVisible = False
-            CONTAINER_MAIN.Size = New System.Drawing.Size(234, 103)
-            CONTAINER_MAIN.TabIndex = 0
-            CONTAINER_MAIN.TopToolStripPanelVisible = False
-            '
-            'TP_MAIN
-            '
-            TP_MAIN.CellBorderStyle = System.Windows.Forms.TableLayoutPanelCellBorderStyle.[Single]
-            TP_MAIN.ColumnCount = 1
-            TP_MAIN.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Percent, 100.0!))
-            TP_MAIN.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Absolute, 20.0!))
-            TP_MAIN.Controls.Add(Me.CH_DOWN_TIME, 0, 0)
-            TP_MAIN.Controls.Add(Me.CH_DOWN_TAG, 0, 1)
-            TP_MAIN.Controls.Add(Me.CH_DOWN_SAVED, 0, 2)
-            TP_MAIN.Dock = System.Windows.Forms.DockStyle.Fill
-            TP_MAIN.Location = New System.Drawing.Point(0, 0)
-            TP_MAIN.Name = "TP_MAIN"
-            TP_MAIN.RowCount = 4
-            TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!))
-            TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!))
-            TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!))
-            TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Percent, 100.0!))
-            TP_MAIN.Size = New System.Drawing.Size(234, 78)
-            TP_MAIN.TabIndex = 0
-            '
-            'CH_DOWN_TIME
-            '
-            Me.CH_DOWN_TIME.AutoSize = True
-            Me.CH_DOWN_TIME.Dock = System.Windows.Forms.DockStyle.Fill
-            Me.CH_DOWN_TIME.Location = New System.Drawing.Point(4, 4)
-            Me.CH_DOWN_TIME.Name = "CH_DOWN_TIME"
-            Me.CH_DOWN_TIME.Size = New System.Drawing.Size(226, 19)
-            Me.CH_DOWN_TIME.TabIndex = 0
-            Me.CH_DOWN_TIME.Text = "Download Timeline"
-            Me.CH_DOWN_TIME.UseVisualStyleBackColor = True
-            '
-            'CH_DOWN_TAG
-            '
-            Me.CH_DOWN_TAG.AutoSize = True
-            Me.CH_DOWN_TAG.Dock = System.Windows.Forms.DockStyle.Fill
-            Me.CH_DOWN_TAG.Location = New System.Drawing.Point(4, 30)
-            Me.CH_DOWN_TAG.Name = "CH_DOWN_TAG"
-            Me.CH_DOWN_TAG.Size = New System.Drawing.Size(226, 19)
-            Me.CH_DOWN_TAG.TabIndex = 1
-            Me.CH_DOWN_TAG.Text = "Download Stories and Tagged data"
-            Me.CH_DOWN_TAG.UseVisualStyleBackColor = True
-            '
-            'CH_DOWN_SAVED
-            '
-            Me.CH_DOWN_SAVED.AutoSize = True
-            Me.CH_DOWN_SAVED.Dock = System.Windows.Forms.DockStyle.Fill
-            Me.CH_DOWN_SAVED.Location = New System.Drawing.Point(4, 56)
-            Me.CH_DOWN_SAVED.Name = "CH_DOWN_SAVED"
-            Me.CH_DOWN_SAVED.Size = New System.Drawing.Size(226, 19)
-            Me.CH_DOWN_SAVED.TabIndex = 2
-            Me.CH_DOWN_SAVED.Text = "Download saved posts"
-            Me.CH_DOWN_SAVED.UseVisualStyleBackColor = True
-            '
-            'AdditionalSettingsForm
-            '
-            Me.AutoScaleDimensions = New System.Drawing.SizeF(6.0!, 13.0!)
-            Me.AutoScaleMode = System.Windows.Forms.AutoScaleMode.Font
-            Me.ClientSize = New System.Drawing.Size(234, 103)
-            Me.Controls.Add(CONTAINER_MAIN)
-            Me.FormBorderStyle = System.Windows.Forms.FormBorderStyle.FixedSingle
-            Me.Icon = Global.SCrawler.My.Resources.SiteResources.InstagramIcon_32
-            Me.KeyPreview = True
-            Me.MaximizeBox = False
-            Me.MaximumSize = New System.Drawing.Size(250, 142)
-            Me.MinimizeBox = False
-            Me.MinimumSize = New System.Drawing.Size(250, 142)
-            Me.Name = "AdditionalSettingsForm"
-            Me.ShowInTaskbar = False
-            Me.SizeGripStyle = System.Windows.Forms.SizeGripStyle.Hide
-            Me.Text = "Additional settings"
-            CONTAINER_MAIN.ContentPanel.ResumeLayout(False)
-            CONTAINER_MAIN.ResumeLayout(False)
-            CONTAINER_MAIN.PerformLayout()
-            TP_MAIN.ResumeLayout(False)
-            TP_MAIN.PerformLayout()
-            Me.ResumeLayout(False)
-
-        End Sub
-        Private WithEvents CH_DOWN_TIME As CheckBox
-        Private WithEvents CH_DOWN_TAG As CheckBox
-        Private WithEvents CH_DOWN_SAVED As CheckBox
-    End Class
-End Namespace

SCrawler/API/Instagram/AdditionalSettingsForm.vb

@@ -1,41 +0,0 @@
-' Copyright (C) 2023  Andy https://github.com/AAndyProgram
-' This program is free software: you can redistribute it and/or modify
-' it under the terms of the GNU General Public License as published by
-' the Free Software Foundation, either version 3 of the License, or
-' (at your option) any later version.
-'
-' This program is distributed in the hope that it will be useful,
-' but WITHOUT ANY WARRANTY
-Imports PersonalUtilities.Forms
-Namespace API.Instagram
-    Friend Class AdditionalSettingsForm
-        Private WithEvents MyDefs As DefaultFormOptions
-        Friend Property MyParameters As SettingsExchangeOptions
-        Friend Sub New(ByVal Parameters As SettingsExchangeOptions)
-            InitializeComponent()
-            MyParameters = Parameters
-            MyDefs = New DefaultFormOptions(Me, Settings.Design)
-        End Sub
-        Private Sub MyForm_Load(sender As Object, e As EventArgs) Handles Me.Load
-            With MyDefs
-                .MyViewInitialize(True)
-                .AddOkCancelToolbar()
-                With MyParameters
-                    CH_DOWN_TIME.Checked = .DownloadTimeline
-                    CH_DOWN_TAG.Checked = .DownloadStoriesTagged
-                    CH_DOWN_SAVED.Checked = .DownloadSaved
-                End With
-                .EndLoaderOperations()
-            End With
-        End Sub
-        Private Sub MyDefs_ButtonOkClick(ByVal Sender As Object, ByVal e As KeyHandleEventArgs) Handles MyDefs.ButtonOkClick
-            MyParameters = New SettingsExchangeOptions With {
-                .DownloadTimeline = CH_DOWN_TIME.Checked,
-                .DownloadStoriesTagged = CH_DOWN_TAG.Checked,
-                .DownloadSaved = CH_DOWN_SAVED.Checked,
-                .Changed = True
-            }
-            MyDefs.CloseForm()
-        End Sub
-    End Class
-End Namespace

SCrawler/API/Instagram/Declarations.vb

@@ -7,11 +7,49 @@
 ' This program is distributed in the hope that it will be useful,
 ' but WITHOUT ANY WARRANTY
 Imports PersonalUtilities.Functions.RegularExpressions
+Imports PersonalUtilities.Tools.Web.Clients
+Imports PersonalUtilities.Tools.Web.Clients.EventArguments
+Imports PersonalUtilities.Tools.Web.Cookies
 Namespace API.Instagram
     Friend Module Declarations
         Friend Const InstagramSite As String = "Instagram"
         Friend Const InstagramSiteKey As String = "AndyProgram_Instagram"
         Friend ReadOnly FilesPattern As RParams = RParams.DMS(".+?([^/\?]+?\.[\w\d]{3,4})(?=(\?|\Z))", 1, EDP.ReturnValue)
         Friend ReadOnly Property DateProvider As New CustomProvider(Function(v, d, p, n, e) ADateTime.ParseUnicode(v))
+        Friend Sub UpdateResponser(ByVal Source As IResponse, ByRef Destination As Responser)
+            Const r_wwwClaimName$ = "x-ig-set-www-claim"
+            Const r_tokenName$ = "csrftoken"
+            If Not Source Is Nothing Then
+                Dim isInternal As Boolean = TypeOf Source Is WebDataResponse
+                Dim wwwClaimName$, tokenName$
+                If isInternal Then
+                    wwwClaimName = r_wwwClaimName
+                    tokenName = r_tokenName
+                Else
+                    wwwClaimName = SiteSettings.Header_IG_WWW_CLAIM
+                    tokenName = SiteSettings.Header_CSRF_TOKEN
+                End If
+                Dim wwwClaim$ = String.Empty
+                Dim token$ = String.Empty
+                With Source
+                    If isInternal Then
+                        If .HeadersExists Then wwwClaim = .Headers.Value(wwwClaimName)
+                        If .CookiesExists Then token = If(.Cookies.FirstOrDefault(Function(c) c.Name = tokenName)?.Value, String.Empty)
+                    Else
+                        If .HeadersExists Then
+                            wwwClaim = .Headers.Value(wwwClaimName)
+                            token = .Headers.Value(tokenName)
+                        End If
+                    End If
+                End With
+
+                If Not wwwClaim.IsEmptyString Then Destination.Headers.Add(SiteSettings.Header_IG_WWW_CLAIM, wwwClaim)
+                If Not token.IsEmptyString Then Destination.Headers.Add(SiteSettings.Header_CSRF_TOKEN, token)
+                If Not isInternal Then
+                    Destination.Cookies.Update(Source.Cookies, CookieKeeper.UpdateModes.ReplaceByNameAll, False, EDP.SendInLog)
+                    Destination.SaveSettings()
+                End If
+            End If
+        End Sub
     End Module
 End Namespace

SCrawler/API/Instagram/EditorExchangeOptions.vb

@@ -9,10 +9,12 @@
 Imports SCrawler.Plugin
 Namespace API.Instagram
     Friend Class EditorExchangeOptions
+        Friend Property GetTimeline As Boolean
         Friend Property GetStories As Boolean
         Friend Property GetTagged As Boolean
         Friend Sub New(ByVal h As ISiteSettings)
             With DirectCast(h, SiteSettings)
+                GetTimeline = CBool(.GetTimeline.Value)
                 GetStories = CBool(.GetStories.Value)
                 GetTagged = CBool(.GetTagged.Value)
             End With

SCrawler/API/Instagram/OptionsForm.Designer.vb

@@ -26,6 +26,7 @@ Namespace API.Instagram
             Dim TP_MAIN As System.Windows.Forms.TableLayoutPanel
             Me.CH_GET_STORIES = New System.Windows.Forms.CheckBox()
             Me.CH_GET_TAGGED = New System.Windows.Forms.CheckBox()
+            Me.CH_GET_TIMELINE = New System.Windows.Forms.CheckBox()
             CONTAINER_MAIN = New System.Windows.Forms.ToolStripContainer()
             TP_MAIN = New System.Windows.Forms.TableLayoutPanel()
             CONTAINER_MAIN.ContentPanel.SuspendLayout()
@@ -39,13 +40,13 @@ Namespace API.Instagram
             'CONTAINER_MAIN.ContentPanel
             '
             CONTAINER_MAIN.ContentPanel.Controls.Add(TP_MAIN)
-            CONTAINER_MAIN.ContentPanel.Size = New System.Drawing.Size(260, 53)
+            CONTAINER_MAIN.ContentPanel.Size = New System.Drawing.Size(260, 79)
             CONTAINER_MAIN.Dock = System.Windows.Forms.DockStyle.Fill
             CONTAINER_MAIN.LeftToolStripPanelVisible = False
             CONTAINER_MAIN.Location = New System.Drawing.Point(0, 0)
             CONTAINER_MAIN.Name = "CONTAINER_MAIN"
             CONTAINER_MAIN.RightToolStripPanelVisible = False
-            CONTAINER_MAIN.Size = New System.Drawing.Size(260, 78)
+            CONTAINER_MAIN.Size = New System.Drawing.Size(260, 104)
             CONTAINER_MAIN.TabIndex = 0
             CONTAINER_MAIN.TopToolStripPanelVisible = False
             '
@@ -54,26 +55,28 @@ Namespace API.Instagram
             TP_MAIN.CellBorderStyle = System.Windows.Forms.TableLayoutPanelCellBorderStyle.[Single]
             TP_MAIN.ColumnCount = 1
             TP_MAIN.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Percent, 100.0!))
-            TP_MAIN.Controls.Add(Me.CH_GET_STORIES, 0, 0)
+            TP_MAIN.Controls.Add(Me.CH_GET_STORIES, 0, 1)
-            TP_MAIN.Controls.Add(Me.CH_GET_TAGGED, 0, 1)
+            TP_MAIN.Controls.Add(Me.CH_GET_TAGGED, 0, 2)
+            TP_MAIN.Controls.Add(Me.CH_GET_TIMELINE, 0, 0)
             TP_MAIN.Dock = System.Windows.Forms.DockStyle.Fill
             TP_MAIN.Location = New System.Drawing.Point(0, 0)
             TP_MAIN.Name = "TP_MAIN"
-            TP_MAIN.RowCount = 3
+            TP_MAIN.RowCount = 4
+            TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!))
             TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!))
             TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!))
             TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Percent, 100.0!))
-            TP_MAIN.Size = New System.Drawing.Size(260, 53)
+            TP_MAIN.Size = New System.Drawing.Size(260, 79)
             TP_MAIN.TabIndex = 0
             '
             'CH_GET_STORIES
             '
             Me.CH_GET_STORIES.AutoSize = True
             Me.CH_GET_STORIES.Dock = System.Windows.Forms.DockStyle.Fill
-            Me.CH_GET_STORIES.Location = New System.Drawing.Point(4, 4)
+            Me.CH_GET_STORIES.Location = New System.Drawing.Point(4, 30)
             Me.CH_GET_STORIES.Name = "CH_GET_STORIES"
             Me.CH_GET_STORIES.Size = New System.Drawing.Size(252, 19)
-            Me.CH_GET_STORIES.TabIndex = 0
+            Me.CH_GET_STORIES.TabIndex = 1
             Me.CH_GET_STORIES.Text = "Get stories"
             Me.CH_GET_STORIES.UseVisualStyleBackColor = True
             '
@@ -81,26 +84,37 @@ Namespace API.Instagram
             '
             Me.CH_GET_TAGGED.AutoSize = True
             Me.CH_GET_TAGGED.Dock = System.Windows.Forms.DockStyle.Fill
-            Me.CH_GET_TAGGED.Location = New System.Drawing.Point(4, 30)
+            Me.CH_GET_TAGGED.Location = New System.Drawing.Point(4, 56)
             Me.CH_GET_TAGGED.Name = "CH_GET_TAGGED"
             Me.CH_GET_TAGGED.Size = New System.Drawing.Size(252, 19)
-            Me.CH_GET_TAGGED.TabIndex = 1
+            Me.CH_GET_TAGGED.TabIndex = 2
             Me.CH_GET_TAGGED.Text = "Get tagged data"
             Me.CH_GET_TAGGED.UseVisualStyleBackColor = True
             '
+            'CH_GET_TIMELINE
+            '
+            Me.CH_GET_TIMELINE.AutoSize = True
+            Me.CH_GET_TIMELINE.Dock = System.Windows.Forms.DockStyle.Fill
+            Me.CH_GET_TIMELINE.Location = New System.Drawing.Point(4, 4)
+            Me.CH_GET_TIMELINE.Name = "CH_GET_TIMELINE"
+            Me.CH_GET_TIMELINE.Size = New System.Drawing.Size(252, 19)
+            Me.CH_GET_TIMELINE.TabIndex = 0
+            Me.CH_GET_TIMELINE.Text = "Get Timeline"
+            Me.CH_GET_TIMELINE.UseVisualStyleBackColor = True
+            '
             'OptionsForm
             '
             Me.AutoScaleDimensions = New System.Drawing.SizeF(6.0!, 13.0!)
             Me.AutoScaleMode = System.Windows.Forms.AutoScaleMode.Font
-            Me.ClientSize = New System.Drawing.Size(260, 78)
+            Me.ClientSize = New System.Drawing.Size(260, 104)
             Me.Controls.Add(CONTAINER_MAIN)
             Me.FormBorderStyle = System.Windows.Forms.FormBorderStyle.FixedSingle
             Me.Icon = Global.SCrawler.My.Resources.SiteResources.InstagramIcon_32
             Me.KeyPreview = True
             Me.MaximizeBox = False
-            Me.MaximumSize = New System.Drawing.Size(276, 117)
+            Me.MaximumSize = New System.Drawing.Size(276, 143)
             Me.MinimizeBox = False
-            Me.MinimumSize = New System.Drawing.Size(276, 117)
+            Me.MinimumSize = New System.Drawing.Size(276, 143)
             Me.Name = "OptionsForm"
             Me.ShowInTaskbar = False
             Me.SizeGripStyle = System.Windows.Forms.SizeGripStyle.Hide
@@ -116,5 +130,6 @@ Namespace API.Instagram
 
         Private WithEvents CH_GET_STORIES As CheckBox
         Private WithEvents CH_GET_TAGGED As CheckBox
+        Private WithEvents CH_GET_TIMELINE As CheckBox
     End Class
 End Namespace

SCrawler/API/Instagram/OptionsForm.vb

@@ -21,6 +21,7 @@ Namespace API.Instagram
                 .MyViewInitialize(True)
                 .AddOkCancelToolbar()
                 With MyExchangeOptions
+                    CH_GET_TIMELINE.Checked = .GetTimeline
                     CH_GET_STORIES.Checked = .GetStories
                     CH_GET_TAGGED.Checked = .GetTagged
                 End With
@@ -29,6 +30,7 @@ Namespace API.Instagram
         End Sub
         Private Sub MyDefs_ButtonOkClick(ByVal Sender As Object, ByVal e As KeyHandleEventArgs) Handles MyDefs.ButtonOkClick
             With MyExchangeOptions
+                .GetTimeline = CH_GET_TIMELINE.Checked
                 .GetStories = CH_GET_STORIES.Checked
                 .GetTagged = CH_GET_TAGGED.Checked
             End With

SCrawler/API/Instagram/SettingsExchangeOptions.vb

@@ -1,23 +0,0 @@
-' Copyright (C) 2023  Andy https://github.com/AAndyProgram
-' This program is free software: you can redistribute it and/or modify
-' it under the terms of the GNU General Public License as published by
-' the Free Software Foundation, either version 3 of the License, or
-' (at your option) any later version.
-'
-' This program is distributed in the hope that it will be useful,
-' but WITHOUT ANY WARRANTY
-Namespace API.Instagram
-    Friend Structure SettingsExchangeOptions
-        Friend DownloadTimeline As Boolean
-        Friend DownloadStoriesTagged As Boolean
-        Friend DownloadSaved As Boolean
-        Friend Changed As Boolean
-        Friend Sub New(ByVal Source As SiteSettings)
-            With Source
-                DownloadTimeline = .DownloadTimeline
-                DownloadStoriesTagged = .DownloadStoriesTagged
-                DownloadSaved = .DownloadSaved
-            End With
-        End Sub
-    End Structure
-End Namespace

SCrawler/API/Instagram/SiteSettings.vb

@@ -13,9 +13,11 @@ Imports PersonalUtilities.Forms
 Imports PersonalUtilities.Functions.XML
 Imports PersonalUtilities.Functions.XML.Base
 Imports PersonalUtilities.Functions.RegularExpressions
+Imports PersonalUtilities.Tools.Web.Clients
+Imports PersonalUtilities.Tools.Web.Cookies
 Imports Download = SCrawler.Plugin.ISiteSettings.Download
 Namespace API.Instagram
-    <Manifest(InstagramSiteKey), SeparatedTasks(1), SavedPosts, SpecialForm(False), SpecialForm(True)>
+    <Manifest(InstagramSiteKey), SeparatedTasks(1), SavedPosts, SpecialForm(False)>
     Friend Class SiteSettings : Inherits SiteSettingsBase
 #Region "Declarations"
 #Region "Images"
@@ -77,28 +79,23 @@ Namespace API.Instagram
         End Class
 #End Region
 #Region "Authorization properties"
-        <PropertyOption(ControlText:="Hash", ControlToolTip:="Instagram session hash", IsAuth:=True, AllowNull:=False), PXML("InstaHash"), ControlNumber(0)>
+        <PropertyOption(ControlText:="Hash", ControlToolTip:="Instagram session hash for tagged posts", IsAuth:=True), PXML("InstaHash"), ControlNumber(0)>
-        Friend ReadOnly Property Hash As PropertyValue
+        Friend ReadOnly Property HashTagged As PropertyValue
-        Private Const HashSavedPosts_Text As String = "Hash 2"
+        <PropertyOption(ControlText:="x-csrftoken", IsAuth:=True, AllowNull:=False), ControlNumber(2)>
-        <PropertyOption(ControlText:=HashSavedPosts_Text, ControlToolTip:="Instagram session hash for saved posts", IsAuth:=True), PXML("InstaHashSavedPosts"), ControlNumber(1)>
-        Friend ReadOnly Property HashSavedPosts As PropertyValue
-        <PropertyOption(ControlText:="x-csrftoken", ControlToolTip:="Instagram token for tagged data", IsAuth:=True, AllowNull:=False), ControlNumber(2)>
         Friend ReadOnly Property CSRF_TOKEN As PropertyValue
         <PropertyOption(ControlText:="x-ig-app-id", IsAuth:=True, AllowNull:=False), ControlNumber(3)>
         Friend Property IG_APP_ID As PropertyValue
         <PropertyOption(ControlText:="x-ig-www-claim", IsAuth:=True, AllowNull:=False), ControlNumber(4)>
         Friend Property IG_WWW_CLAIM As PropertyValue
-        Private Const SavedPostsUserName_Text As String = "Saved posts user"
-        <PropertyOption(ControlText:=SavedPostsUserName_Text, ControlToolTip:="Personal profile username", IsAuth:=True), PXML("SavedPostsUserName"), ControlNumber(5)>
-        Friend ReadOnly Property SavedPostsUserName As PropertyValue
         Friend Overrides Function BaseAuthExists() As Boolean
-            Return If(Responser.Cookies?.Count, 0) > 0 And ACheck(IG_APP_ID.Value) And ACheck(IG_WWW_CLAIM.Value) And ACheck(CSRF_TOKEN.Value)
+            Return Responser.CookiesExists And ACheck(IG_APP_ID.Value) And ACheck(IG_WWW_CLAIM.Value) And ACheck(CSRF_TOKEN.Value)
         End Function
         Private Const Header_IG_APP_ID As String = "x-ig-app-id"
-        Private Const Header_IG_WWW_CLAIM As String = "x-ig-www-claim"
+        Friend Const Header_IG_WWW_CLAIM As String = "x-ig-www-claim"
-        Private Const Header_CSRF_TOKEN As String = "x-csrftoken"
+        Friend Const Header_CSRF_TOKEN As String = "x-csrftoken"
+        Private _FieldsChangerSuspended As Boolean = False
         Private Sub ChangeResponserFields(ByVal PropName As String, ByVal Value As Object)
-            If Not PropName.IsEmptyString Then
+            If Not _FieldsChangerSuspended And Not PropName.IsEmptyString Then
                 Dim f$ = String.Empty
                 Select Case PropName
                     Case NameOf(IG_APP_ID) : f = Header_IG_APP_ID
@@ -106,41 +103,46 @@ Namespace API.Instagram
                     Case NameOf(CSRF_TOKEN) : f = Header_CSRF_TOKEN
                 End Select
                 If Not f.IsEmptyString Then
-                    Responser.HeadersRemove(f)
+                    Responser.Headers.Remove(f)
-                    If Not CStr(Value).IsEmptyString Then Responser.HeadersAdd(f, CStr(Value))
+                    If Not CStr(Value).IsEmptyString Then Responser.Headers.Add(f, CStr(Value))
                     Responser.SaveSettings()
                 End If
             End If
         End Sub
 #End Region
 #Region "Download properties"
-        <PropertyOption(ControlText:="Request timer", AllowNull:=False), PXML("RequestsWaitTimer"), ControlNumber(6)>
+        <PropertyOption(ControlText:="Request timer", AllowNull:=False), PXML("RequestsWaitTimer"), ControlNumber(20)>
         Friend ReadOnly Property RequestsWaitTimer As PropertyValue
         <Provider(NameOf(RequestsWaitTimer), FieldsChecker:=True)>
         Private ReadOnly Property RequestsWaitTimerProvider As IFormatProvider
-        <PropertyOption(ControlText:="Request timer counter", AllowNull:=False, LeftOffset:=120), PXML("RequestsWaitTimerTaskCount"), ControlNumber(7)>
+        <PropertyOption(ControlText:="Request timer counter", AllowNull:=False, LeftOffset:=120), PXML("RequestsWaitTimerTaskCount"), ControlNumber(21)>
         Friend ReadOnly Property RequestsWaitTimerTaskCount As PropertyValue
         <Provider(NameOf(RequestsWaitTimerTaskCount), FieldsChecker:=True)>
         Private ReadOnly Property RequestsWaitTimerTaskCountProvider As IFormatProvider
-        <PropertyOption(ControlText:="Posts limit timer", AllowNull:=False), PXML("SleepTimerOnPostsLimit"), ControlNumber(8)>
+        <PropertyOption(ControlText:="Posts limit timer", AllowNull:=False), PXML("SleepTimerOnPostsLimit"), ControlNumber(22)>
         Friend ReadOnly Property SleepTimerOnPostsLimit As PropertyValue
         <Provider(NameOf(SleepTimerOnPostsLimit), FieldsChecker:=True)>
         Private ReadOnly Property SleepTimerOnPostsLimitProvider As IFormatProvider
-        <PropertyOption(ControlText:="Get stories"), PXML, ControlNumber(9)>
+        <PropertyOption(ControlText:="Get timeline", ControlToolTip:="Default value for new users"), PXML, ControlNumber(23)>
+        Friend ReadOnly Property GetTimeline As PropertyValue
+        <PropertyOption(ControlText:="Get stories", ControlToolTip:="Default value for new users"), PXML, ControlNumber(24)>
         Friend ReadOnly Property GetStories As PropertyValue
-        <PropertyOption(ControlText:="Get tagged photos"), PXML, ControlNumber(10)>
+        <PropertyOption(ControlText:="Get tagged photos", ControlToolTip:="Default value for new users"), PXML, ControlNumber(25)>
         Friend ReadOnly Property GetTagged As PropertyValue
         <PropertyOption(ControlText:="Tagged notify limit",
                         ControlToolTip:="If the number of tagged posts exceeds this number you will be notified." & vbCr &
-                        "-1 to disable"), PXML, ControlNumber(11)>
+                        "-1 to disable"), PXML, ControlNumber(26)>
         Friend ReadOnly Property TaggedNotifyLimit As PropertyValue
         <Provider(NameOf(TaggedNotifyLimit), FieldsChecker:=True)>
         Private ReadOnly Property TaggedNotifyLimitProvider As IFormatProvider
 #End Region
 #Region "Download ready"
-        Friend ReadOnly Property DownloadTimeline As XMLValue(Of Boolean)
+        <PropertyOption(ControlText:="Download timeline", ControlToolTip:="Download timeline"), PXML, ControlNumber(10)>
-        Friend ReadOnly Property DownloadStoriesTagged As XMLValue(Of Boolean)
+        Friend ReadOnly Property DownloadTimeline As PropertyValue
-        Friend ReadOnly Property DownloadSaved As XMLValue(Of Boolean)
+        <PropertyOption(ControlText:="Download stories", ControlToolTip:="Download stories"), PXML, ControlNumber(11)>
+        Friend ReadOnly Property DownloadStories As PropertyValue
+        <PropertyOption(ControlText:="Download tagged", ControlToolTip:="Download tagged posts"), PXML, ControlNumber(12)>
+        Friend ReadOnly Property DownloadTagged As PropertyValue
 #End Region
 #Region "429 bypass"
         Private ReadOnly Property DownloadingErrorDate As XMLValue(Of Date)
@@ -202,29 +204,25 @@ Namespace API.Instagram
 
             With Responser
                 If .Headers.Count > 0 Then
-                    token = .HeadersValue(Header_CSRF_TOKEN)
+                    token = .Headers.Value(Header_CSRF_TOKEN)
-                    app_id = .HeadersValue(Header_IG_APP_ID)
+                    app_id = .Headers.Value(Header_IG_APP_ID)
-                    www_claim = .HeadersValue(Header_IG_WWW_CLAIM)
+                    www_claim = .Headers.Value(Header_IG_WWW_CLAIM)
-                End If
-                If Not .Cookies Is Nothing Then
-                    .Cookies.ChangedAllowInternalDrop = False
-                    .Cookies.Changed = False
                 End If
+                .CookiesExtractMode = Responser.CookiesExtractModes.Response
+                .CookiesUpdateMode = CookieKeeper.UpdateModes.ReplaceByNameAll
+                .CookiesExtractedAutoSave = False
             End With
 
             Dim n() As String = {SettingsCLS.Name_Node_Sites, Site.ToString}
 
-            SavedPostsUserName = New PropertyValue(String.Empty, GetType(String))
+            HashTagged = New PropertyValue(String.Empty, GetType(String))
-
-            Hash = New PropertyValue(String.Empty, GetType(String))
-            HashSavedPosts = New PropertyValue(String.Empty, GetType(String))
             CSRF_TOKEN = New PropertyValue(token, GetType(String), Sub(v) ChangeResponserFields(NameOf(CSRF_TOKEN), v))
             IG_APP_ID = New PropertyValue(app_id, GetType(String), Sub(v) ChangeResponserFields(NameOf(IG_APP_ID), v))
             IG_WWW_CLAIM = New PropertyValue(www_claim, GetType(String), Sub(v) ChangeResponserFields(NameOf(IG_WWW_CLAIM), v))
 
-            DownloadTimeline = New XMLValue(Of Boolean)("DownloadTimeline", True, _XML, n)
+            DownloadTimeline = New PropertyValue(True)
-            DownloadStoriesTagged = New XMLValue(Of Boolean)("DownloadStoriesTagged", True, _XML, n)
+            DownloadStories = New PropertyValue(True)
-            DownloadSaved = New XMLValue(Of Boolean)("DownloadSaved", True, _XML, n)
+            DownloadTagged = New PropertyValue(False)
 
             RequestsWaitTimer = New PropertyValue(1000)
             RequestsWaitTimerProvider = New TimersChecker(100)
@@ -233,6 +231,7 @@ Namespace API.Instagram
             SleepTimerOnPostsLimit = New PropertyValue(60000)
             SleepTimerOnPostsLimitProvider = New TimersChecker(10000)
 
+            GetTimeline = New PropertyValue(True)
             GetStories = New PropertyValue(False)
             GetTagged = New PropertyValue(False)
             TaggedNotifyLimit = New PropertyValue(200)
@@ -249,90 +248,8 @@ Namespace API.Instagram
             UserRegex = RParams.DMS("[htps:/]{7,8}.*?instagram.com/([^/]+)", 1)
             ImageVideoContains = "instagram.com"
         End Sub
-        Private Structure LatestValues
-            Friend Hash As String
-            Friend Hash2 As String
-            Friend Token As String
-            Friend AppID As String
-            Friend WwwClaim As String
-            Friend Exists As Boolean
-            Friend Sub New(ByVal Source As SiteSettings)
-                Exists = True
-                With Source
-                    Hash = AConvert(Of String)(.Hash.Value, String.Empty)
-                    Hash2 = AConvert(Of String)(.HashSavedPosts.Value, String.Empty)
-                    With .Responser
-                        Token = .HeadersValue(Header_CSRF_TOKEN)
-                        AppID = .HeadersValue(Header_IG_APP_ID)
-                        WwwClaim = .HeadersValue(Header_IG_WWW_CLAIM)
-                    End With
-                End With
-            End Sub
-        End Structure
-        Private LV As LatestValues = Nothing
-        Private ASO As SettingsExchangeOptions = Nothing
-        Friend Overrides Sub BeginEdit()
-            LV = New LatestValues(Me)
-            ASO = Nothing
-            MyBase.BeginEdit()
-        End Sub
-        Friend Overrides Sub EndEdit()
-            LV = Nothing
-            ASO = Nothing
-            MyBase.EndEdit()
-        End Sub
-        Friend Overrides Sub Update()
-            If LV.Exists Then
-                Dim __lv As New LatestValues(Me)
-                If If(Responser.Cookies?.Count, 0) > 0 Then
-                    Dim _cookiesChanged As Boolean = If(Responser.Cookies?.Changed, False)
-                    If Not DownloadTimeline AndAlso (_cookiesChanged Or
-                       (Not LV.Hash = __lv.Hash And Not __lv.Hash.IsEmptyString)) Then DownloadTimeline.Value = True
-                    If Not DownloadSaved AndAlso (_cookiesChanged Or (Not LV.Hash2 = __lv.Hash2 And Not __lv.Hash2.IsEmptyString)) Then DownloadSaved.Value = True
-                    If Not DownloadStoriesTagged AndAlso (
-                           _cookiesChanged Or (
-                                (Not LV.Hash = __lv.Hash Or Not LV.Token = __lv.Token Or Not LV.AppID = __lv.AppID Or Not LV.WwwClaim = __lv.WwwClaim) And
-                                (Not __lv.Hash.IsEmptyString And Not __lv.Token.IsEmptyString And Not __lv.AppID.IsEmptyString And Not __lv.WwwClaim.IsEmptyString)
-                            )) Then DownloadStoriesTagged.Value = True
-                End If
-            End If
-            If ASO.Changed Then
-                DownloadTimeline.Value = ASO.DownloadTimeline
-                DownloadStoriesTagged.Value = ASO.DownloadStoriesTagged
-                DownloadSaved.Value = ASO.DownloadSaved
-            End If
-            LV = Nothing
-            ASO = Nothing
-            If Not Responser.Cookies Is Nothing Then Responser.Cookies.Changed = False
-            MyBase.Update()
-        End Sub
 #End Region
 #Region "PropertiesDataChecker"
-        <PropertiesDataChecker({NameOf(Hash), NameOf(HashSavedPosts)})>
-        Private Function CheckHashControls(ByVal p As IEnumerable(Of PropertyData)) As Boolean
-            If p.ListExists(2) Then
-                Dim h$ = String.Empty
-                Dim hsp$ = String.Empty
-                For Each pp As PropertyData In p
-                    Select Case pp.Name
-                        Case NameOf(Hash) : h = AConvert(Of String)(pp.Value, String.Empty)
-                        Case NameOf(HashSavedPosts) : hsp = AConvert(Of String)(pp.Value, String.Empty)
-                    End Select
-                Next
-                If h.IsEmptyString And hsp.IsEmptyString Then
-                    Return True
-                Else
-                    If h = hsp Then
-                        MsgBoxE({"InstaHash for saved posts must be different from InstaHash!", "InstaHash are equal"}, vbCritical)
-                        Return False
-                    Else
-                        Return True
-                    End If
-                End If
-            Else
-                Return False
-            End If
-        End Function
         <PropertiesDataChecker({NameOf(TaggedNotifyLimit)})>
         Private Function CheckNotifyLimit(ByVal p As IEnumerable(Of PropertyData)) As Boolean
             If p.ListExists Then
@@ -351,37 +268,6 @@ Namespace API.Instagram
             End If
             Return False
         End Function
-        <PropertiesDataChecker({NameOf(HashSavedPosts), NameOf(SavedPostsUserName)})>
-        Private Function CheckSavedOptions(ByVal p As IEnumerable(Of PropertyData)) As Boolean
-            If p.ListExists Then
-                Const MsgTitle$ = "Saved posts credentials"
-                Dim __hash$ = String.Empty
-                Dim __name$ = String.Empty
-                Dim _OptionlErrorText$ = $"For download saved posts, you must to set both [{HashSavedPosts_Text}] and [{SavedPostsUserName_Text}]."
-                For i% = 0 To p.Count - 1
-                    Select Case p(i).Name
-                        Case NameOf(HashSavedPosts) : __hash = p(i).Value
-                        Case NameOf(SavedPostsUserName) : __name = p(i).Value
-                    End Select
-                Next
-                If __hash = __name Then
-                    If __hash.IsEmptyString Then
-                        Return True
-                    Else
-                        MsgBoxE({$"[{HashSavedPosts_Text}] and [{SavedPostsUserName_Text}] for saved posts cannot be equal!", MsgTitle}, vbCritical)
-                    End If
-                Else
-                    If __hash.IsEmptyString Then
-                        MsgBoxE({$"[{HashSavedPosts_Text}] not set.{vbCr}{_OptionlErrorText}", MsgTitle}, vbCritical)
-                    ElseIf __name.IsEmptyString Then
-                        MsgBoxE({$"[{SavedPostsUserName_Text}] not set.{vbCr}{_OptionlErrorText}", MsgTitle}, vbCritical)
-                    Else
-                        Return True
-                    End If
-                End If
-            End If
-            Return False
-        End Function
 #End Region
 #Region "Plugin functions"
         Friend Overrides Function GetInstance(ByVal What As Download) As IPluginContentProvider
@@ -389,7 +275,7 @@ Namespace API.Instagram
                 Case Download.Main : Return New UserData
                 Case Download.SavedPosts
                     Dim u As New UserData
-                    DirectCast(u, UserDataBase).User = New UserInfo With {.Name = CStr(AConvert(Of String)(SavedPostsUserName.Value, String.Empty))}
+                    DirectCast(u, UserDataBase).User = New UserInfo With {.Name = Site}
                     Return u
             End Select
             Return Nothing
@@ -397,13 +283,7 @@ Namespace API.Instagram
 #Region "Downloading"
         Friend Property SkipUntilNextSession As Boolean = False
         Friend Overrides Function ReadyToDownload(ByVal What As Download) As Boolean
-            If ActiveJobs < 2 AndAlso Not SkipUntilNextSession AndAlso ReadyForDownload AndAlso BaseAuthExists() Then
+            Return ActiveJobs < 2 AndAlso Not SkipUntilNextSession AndAlso ReadyForDownload AndAlso BaseAuthExists() AndAlso DownloadTimeline.Value
-                Select Case What
-                    Case Download.Main : Return ACheck(Hash.Value) And DownloadTimeline
-                    Case Download.SavedPosts : Return ACheck(HashSavedPosts.Value) And DownloadSaved
-                End Select
-            End If
-            Return False
         End Function
         Private ActiveJobs As Integer = 0
         Private _NextWNM As UserData.WNM = UserData.WNM.Notify
@@ -432,6 +312,10 @@ Namespace API.Instagram
                 _NextTagged = .TaggedCheckSession
                 LastDownloadDate.Value = Now
                 LastRequestsCount.Value = .RequestsCount
+                _FieldsChangerSuspended = True
+                IG_WWW_CLAIM.Value = Responser.Headers.Value(Header_IG_WWW_CLAIM)
+                CSRF_TOKEN.Value = Responser.Headers.Value(Header_CSRF_TOKEN)
+                _FieldsChangerSuspended = False
             End With
         End Sub
         Friend Overrides Sub DownloadDone(ByVal What As Download)
@@ -451,12 +335,14 @@ Namespace API.Instagram
                 Using f As New OptionsForm(Options) : f.ShowDialog() : End Using
             End If
         End Sub
-        Friend Overrides Sub OpenSettingsForm()
+        Friend Overrides Function GetUserPostUrl(ByVal User As UserDataBase, ByVal Media As UserMedia) As String
-            Using f As New AdditionalSettingsForm(If(ASO.Changed, ASO, New SettingsExchangeOptions(Me)))
+            Try
-                f.ShowDialog()
+                Dim code$ = DirectCast(User, UserData).GetPostCodeById(Media.Post.ID)
-                If f.DialogResult = DialogResult.OK Then ASO = f.MyParameters
+                If Not code.IsEmptyString Then Return $"https://instagram.com/p/{code}/" Else Return String.Empty
-            End Using
+            Catch ex As Exception
-        End Sub
+                Return ErrorsDescriber.Execute(EDP.SendInLog, ex, "Can't open user's post", String.Empty)
+            End Try
+        End Function
 #End Region
     End Class
 End Namespace

SCrawler/API/Instagram/UserData.vb

@@ -9,6 +9,7 @@
 Imports System.Net
 Imports System.Threading
 Imports PersonalUtilities.Functions.XML
+Imports PersonalUtilities.Functions.XML.Base
 Imports PersonalUtilities.Functions.Messaging
 Imports PersonalUtilities.Functions.RegularExpressions
 Imports PersonalUtilities.Tools.Web.Clients
@@ -20,29 +21,68 @@ Namespace API.Instagram
 #Region "XML Names"
         Private Const Name_LastCursor As String = "LastCursor"
         Private Const Name_FirstLoadingDone As String = "FirstLoadingDone"
+        Private Const Name_GetTimeline As String = "GetTimeline"
         Private Const Name_GetStories As String = "GetStories"
         Private Const Name_GetTagged As String = "GetTaggedData"
         Private Const Name_TaggedChecked As String = "TaggedChecked"
 #End Region
 #Region "Declarations"
+        Private Structure PostKV : Implements IEContainerProvider
+            Private Const Name_Code As String = "Code"
+            Private Const Name_Section As String = "Section"
+            Friend Code As String
+            Friend ID As String
+            Friend Section As Sections
+            Friend Sub New(ByVal _Section As Sections)
+                Section = _Section
+            End Sub
+            Friend Sub New(ByVal _Code As String, ByVal _ID As String, ByVal _Section As Sections)
+                Code = _Code
+                ID = _ID
+                Section = _Section
+            End Sub
+            Private Sub New(ByVal e As EContainer)
+                Code = e.Attribute(Name_Code)
+                Section = e.Attribute(Name_Section)
+                ID = e.Value
+            End Sub
+            Public Shared Widening Operator CType(ByVal e As EContainer) As PostKV
+                Return New PostKV(e)
+            End Operator
+            Public Overrides Function Equals(ByVal Obj As Object) As Boolean
+                If Not IsNothing(Obj) AndAlso TypeOf Obj Is PostKV Then
+                    With DirectCast(Obj, PostKV)
+                        Return Code = .Code And ID = .ID And Section = .Section
+                    End With
+                Else
+                    Return False
+                End If
+            End Function
+            Private Function ToEContainer(Optional ByVal e As ErrorsDescriber = Nothing) As EContainer Implements IEContainerProvider.ToEContainer
+                Return New EContainer("Post", ID, {New EAttribute(Name_Section, CInt(Section)), New EAttribute(Name_Code, Code)})
+            End Function
+        End Structure
         Private ReadOnly Property MySiteSettings As SiteSettings
             Get
                 Return DirectCast(HOST.Source, SiteSettings)
             End Get
         End Property
-        Private ReadOnly _SavedPostsIDs As New List(Of String)
+        Private ReadOnly PostsKVIDs As List(Of PostKV)
+        Private ReadOnly PostsToReparse As List(Of PostKV)
         Private LastCursor As String = String.Empty
         Private FirstLoadingDone As Boolean = False
+        Friend Property GetTimeline As Boolean = True
         Friend Property GetStories As Boolean
         Friend Property GetTaggedData As Boolean
 #End Region
 #Region "Exchange options"
         Friend Overrides Function ExchangeOptionsGet() As Object
-            Return New EditorExchangeOptions(HOST.Source) With {.GetStories = GetStories, .GetTagged = GetTaggedData}
+            Return New EditorExchangeOptions(HOST.Source) With {.GetTimeline = GetTimeline, .GetStories = GetStories, .GetTagged = GetTaggedData}
         End Function
         Friend Overrides Sub ExchangeOptionsSet(ByVal Obj As Object)
             If Not Obj Is Nothing AndAlso TypeOf Obj Is EditorExchangeOptions Then
                 With DirectCast(Obj, EditorExchangeOptions)
+                    GetTimeline = .GetTimeline
                     GetStories = .GetStories
                     GetTaggedData = .GetTagged
                 End With
@@ -51,17 +91,21 @@ Namespace API.Instagram
 #End Region
 #Region "Initializer, loader"
         Friend Sub New()
+            PostsKVIDs = New List(Of PostKV)
+            PostsToReparse = New List(Of PostKV)
         End Sub
         Protected Overrides Sub LoadUserInformation_OptionalFields(ByRef Container As XmlFile, ByVal Loading As Boolean)
             If Loading Then
                 LastCursor = Container.Value(Name_LastCursor)
                 FirstLoadingDone = Container.Value(Name_FirstLoadingDone).FromXML(Of Boolean)(False)
+                GetTimeline = Container.Value(Name_GetTimeline).FromXML(Of Boolean)(CBool(MySiteSettings.GetTimeline.Value))
                 GetStories = Container.Value(Name_GetStories).FromXML(Of Boolean)(CBool(MySiteSettings.GetStories.Value))
                 GetTaggedData = Container.Value(Name_GetTagged).FromXML(Of Boolean)(CBool(MySiteSettings.GetTagged.Value))
                 TaggedChecked = Container.Value(Name_TaggedChecked).FromXML(Of Boolean)(False)
             Else
                 Container.Add(Name_LastCursor, LastCursor)
                 Container.Add(Name_FirstLoadingDone, FirstLoadingDone.BoolToInteger)
+                Container.Add(Name_GetTimeline, GetTimeline.BoolToInteger)
                 Container.Add(Name_GetStories, GetStories.BoolToInteger)
                 Container.Add(Name_GetTagged, GetTaggedData.BoolToInteger)
                 Container.Add(Name_TaggedChecked, TaggedChecked.BoolToInteger)
@@ -78,48 +122,128 @@ Namespace API.Instagram
                 End If
                 Throw New ExitException
             End Sub
-            Friend Sub New()
-            End Sub
-            Friend Sub New(ByRef CompleteArg As Boolean)
-                CompleteArg = True
-            End Sub
         End Class
+        Private Sub LoadSavePostsKV(ByVal Load As Boolean)
+            Dim x As XmlFile
+            Dim f As SFile = MyFilePosts
+            If Not f.IsEmptyString Then
+                f.Name &= "_KV"
+                f.Extension = "xml"
+                If Load Then
+                    PostsKVIDs.Clear()
+                    x = New XmlFile(f, Protector.Modes.All, False) With {.AllowSameNames = True, .XmlReadOnly = True}
+                    x.LoadData()
+                    If x.Count > 0 Then PostsKVIDs.ListAddList(x, LAP.IgnoreICopier)
+                    x.Dispose()
+                Else
+                    x = New XmlFile With {.AllowSameNames = True}
+                    x.AddRange(PostsKVIDs)
+                    x.Name = "Posts"
+                    x.Save(f, EDP.SendInLog)
+                    x.Dispose()
+                End If
+            End If
+        End Sub
+        Private Overloads Function PostKvExists(ByVal pkv As PostKV) As Boolean
+            Return PostKvExists(pkv.ID, False, pkv.Section) OrElse PostKvExists(pkv.Code, True, pkv.Section)
+        End Function
+        Private Overloads Function PostKvExists(ByVal PostCodeId As String, ByVal IsCode As Boolean, ByVal Section As Sections) As Boolean
+            If Not PostCodeId.IsEmptyString And PostsKVIDs.Count > 0 Then
+                If PostsKVIDs.FindIndex(Function(p) p.Section = Section AndAlso If(IsCode, p.Code = PostCodeId, p.ID = PostCodeId)) >= 0 Then
+                    Return True
+                ElseIf Not IsCode Then
+                    Return _TempPostsList.Contains(GetPostIdBySection(PostCodeId, Section)) Or
+                           _TempPostsList.Contains(PostCodeId.Replace($"_{ID}", String.Empty)) Or
+                           _TempPostsList.Contains(GetPostIdBySection(PostCodeId.Replace($"_{ID}", String.Empty), Section))
+                End If
+            End If
+            Return False
+        End Function
+        Friend Function GetPostCodeById(ByVal PostID As String) As String
+            Try
+                If Not PostID.IsEmptyString Then
+                    Dim f As SFile = MyFilePosts
+                    If Not f.IsEmptyString Then
+                        f.Name &= "_KV"
+                        f.Extension = "xml"
+                        Dim l As List(Of PostKV) = Nothing
+                        Using x As New XmlFile(f, Protector.Modes.All, False) With {.AllowSameNames = True, .XmlReadOnly = True}
+                            x.LoadData()
+                            l.ListAddList(x, LAP.IgnoreICopier)
+                        End Using
+                        Dim code$ = String.Empty
+                        If l.ListExists Then
+                            Dim i% = l.FindIndex(Function(p) p.ID = PostID)
+                            If i >= 0 Then code = l(i).Code
+                            l.Clear()
+                        End If
+                        Return code
+                    End If
+                End If
+                Return String.Empty
+            Catch ex As Exception
+                Return ErrorsDescriber.Execute(EDP.SendInLog, ex, $"{ToStringForLog()}: Cannot find post code by ID ({PostID})", String.Empty)
+            End Try
+        End Function
+        Private Function GetPostIdBySection(ByVal ID As String, ByVal Section As Sections) As String
+            If Section = Sections.Timeline Then
+                Return ID
+            Else
+                Return $"{Section}_{ID}"
+            End If
+        End Function
+        Private _DownloadingInProgress As Boolean = False
         Protected Overrides Sub DownloadDataF(ByVal Token As CancellationToken)
             Dim s As Sections = Sections.Timeline
+            Dim errorFound As Boolean = False
             Try
+                LoadSavePostsKV(True)
+                _DownloadingInProgress = True
+                AddHandler Responser.ResponseReceived, AddressOf Responser_ResponseReceived
                 ThrowAny(Token)
-                _InstaHash = String.Empty
                 HasError = False
-                Dim fc As Boolean = IIf(IsSavedPosts, MySiteSettings.DownloadSaved.Value, MySiteSettings.DownloadTimeline.Value)
+                Dim dt As Func(Of Boolean) = Function() (CBool(MySiteSettings.DownloadTimeline.Value) And GetTimeline) Or IsSavedPosts
-                If fc And Not LastCursor.IsEmptyString Then
+                If dt.Invoke And Not LastCursor.IsEmptyString Then
                     s = IIf(IsSavedPosts, Sections.SavedPosts, Sections.Timeline)
                     DownloadData(LastCursor, s, Token)
                     ThrowAny(Token)
                     If Not HasError Then FirstLoadingDone = True
                 End If
-                If fc And Not HasError Then
+                If dt.Invoke And Not HasError Then
                     s = IIf(IsSavedPosts, Sections.SavedPosts, Sections.Timeline)
                     DownloadData(String.Empty, s, Token)
                     ThrowAny(Token)
                     If Not HasError Then FirstLoadingDone = True
                 End If
                 If FirstLoadingDone Then LastCursor = String.Empty
-                If IsSavedPosts Then
+                If Not IsSavedPosts AndAlso MySiteSettings.BaseAuthExists() Then
-                    If MySiteSettings.DownloadSaved Then s = Sections.SavedPosts : DownloadPosts(Token)
+                    If CBool(MySiteSettings.DownloadStories.Value) And GetStories Then s = Sections.Stories : DownloadData(String.Empty, s, Token)
-                ElseIf MySiteSettings.BaseAuthExists() Then
+                    If CBool(MySiteSettings.DownloadTagged.Value) And ACheck(MySiteSettings.HashTagged.Value) And GetTaggedData Then s = Sections.Tagged : DownloadData(String.Empty, s, Token)
-                    DownloadedTags = 0
-                    If MySiteSettings.DownloadStoriesTagged And GetStories Then s = Sections.Stories : DownloadData(String.Empty, s, Token)
-                    If MySiteSettings.DownloadStoriesTagged And GetTaggedData Then s = Sections.Tagged : DownloadData(String.Empty, s, Token)
                 End If
                 If WaitNotificationMode = WNM.SkipTemp Or WaitNotificationMode = WNM.SkipCurrent Then WaitNotificationMode = WNM.Notify
             Catch eex As ExitException
             Catch ex As Exception
-                ProcessException(ex, Token, "[API.Instagram.UserData.DownloadDataF]", False, s)
+                errorFound = True
+                Throw ex
             Finally
                 E560Thrown = False
+                UpdateResponser()
+                If Not errorFound Then LoadSavePostsKV(False)
             End Try
         End Sub
-        Private _InstaHash As String = String.Empty
+        Private Sub UpdateResponser()
+            Try
+                If _DownloadingInProgress AndAlso Not Responser Is Nothing AndAlso Not Responser.Disposed Then
+                    _DownloadingInProgress = False
+                    RemoveHandler Responser.ResponseReceived, AddressOf Responser_ResponseReceived
+                    Declarations.UpdateResponser(Responser, MySiteSettings.Responser)
+                End If
+            Catch
+            End Try
+        End Sub
+        Private Sub Responser_ResponseReceived(ByVal Sender As Object, ByVal e As EventArguments.WebDataResponse)
+            Declarations.UpdateResponser(e, Responser)
+        End Sub
         Private Enum Sections : Timeline : Tagged : Stories : SavedPosts : End Enum
         Private Const StoriesFolder As String = "Stories"
         Private Const TaggedFolder As String = "Tagged"
@@ -145,7 +269,7 @@ Namespace API.Instagram
                                               "What do you want to do?", "Waiting for Instagram download...",
                                               {
                                                New MsgBoxButton("Wait") With {.ToolTip = "Wait and ask again when the error is found."},
-                                               New MsgBoxButton("Wait (disable current") With {.ToolTip = "Wait and skip future prompts while downloading the current profile."},
+                                               New MsgBoxButton("Wait (disable current)") With {.ToolTip = "Wait and skip future prompts while downloading the current profile."},
                                                New MsgBoxButton("Abort") With {.ToolTip = "Abort operation"},
                                                New MsgBoxButton("Wait (disable all)") With {.ToolTip = "Wait and skip future prompts while downloading the current session."}
                                               },
@@ -180,13 +304,11 @@ Namespace API.Instagram
 #Region "Tags"
         Private TaggedChecked As Boolean = False
         Friend TaggedCheckSession As Boolean = True
-        Private DownloadedTags As Integer = 0
         Private DownloadTagsLimit As Integer? = Nothing
-        Private ReadOnly Property TaggedLimitsNotifications(Optional ByVal v As Integer? = Nothing) As Boolean
+        Private ReadOnly Property TaggedLimitsNotifications(ByVal v As Integer) As Boolean
             Get
                 Return Not TaggedChecked AndAlso TaggedCheckSession AndAlso
-                       CInt(MySiteSettings.TaggedNotifyLimit.Value) > 0 AndAlso
+                       CInt(MySiteSettings.TaggedNotifyLimit.Value) > 0 AndAlso v > CInt(MySiteSettings.TaggedNotifyLimit.Value)
-                       (Not v.HasValue OrElse v.Value > CInt(MySiteSettings.TaggedNotifyLimit.Value))
             End Get
         End Property
         Private Function SetTagsLimit(ByVal Max As Integer, ByVal p As ANumbers) As DialogResult
@@ -219,8 +341,9 @@ Namespace API.Instagram
         End Function
         Private Function TaggedContinue(ByVal TaggedCount As Integer) As DialogResult
             Dim agi As New ANumbers With {.FormatOptions = ANumbers.Options.GroupIntegral}
-            Dim msg As New MMessage($"The number of tagged posts by user [{ToString()}] is {TaggedCount.NumToString(agi)}" & vbCr &
+            Dim msg As New MMessage($"The number of already downloaded tagged posts by user [{ToString()}] is {TaggedCount.NumToString(agi)}" & vbCr &
-                                    $"This is about {(TaggedCount / 12).RoundUp.NumToString(agi)} requests." & vbCr &
+                                    "There is currently no way to know how many posts exist." & vbCr &
+                                    "One request will be spent per post." & vbCr &
                                     "The tagged data download operation can take a long time.",
                                     "Too much tagged data",
                                     {
@@ -252,40 +375,45 @@ Namespace API.Instagram
             Dim URL$ = String.Empty
             Dim StoriesList As List(Of String) = Nothing
             Dim StoriesRequested As Boolean = False
-            Dim _DownloadComplete As Boolean = False
+            Dim dValue% = 1
             LastCursor = Cursor
             Try
-                Do While Not _DownloadComplete
+                Do While dValue = 1
                     ThrowAny(Token)
                     If Not Ready() Then Thread.Sleep(10000) : ThrowAny(Token) : Continue Do
                     ReconfigureAwaiter()
 
                     Try
-                        Dim n As EContainer, nn As EContainer, node As EContainer
+                        Dim n As EContainer, nn As EContainer
                         Dim HasNextPage As Boolean = False
-                        Dim Pinned As Boolean
                         Dim EndCursor$ = String.Empty
                         Dim PostID$ = String.Empty, PostDate$ = String.Empty, SpecFolder$ = String.Empty
-                        Dim TaggedCount%
+                        Dim PostIDKV As PostKV
                         Dim ENode() As Object = Nothing
                         NextRequest(True)
 
                         'Check environment
-                        If Cursor.IsEmptyString And _InstaHash.IsEmptyString Then _
+                        If Not IsSavedPosts Then
-                           _InstaHash = CStr(If(IsSavedPosts, MySiteSettings.HashSavedPosts, MySiteSettings.Hash).Value)
+                            If ID.IsEmptyString Then GetUserId()
-                        If ID.IsEmptyString Then GetUserId()
+                            If ID.IsEmptyString Then Throw New ArgumentException("User ID is not detected", "ID")
-                        If ID.IsEmptyString Then Throw New ArgumentException("User ID is not detected", "ID")
+                        End If
 
                         'Create query
                         Select Case Section
-                            Case Sections.Timeline, Sections.SavedPosts
+                            Case Sections.Timeline
+                                URL = $"https://www.instagram.com/api/v1/feed/user/{Name}/username/?count=50" &
+                                        If(Cursor.IsEmptyString, String.Empty, $"&max_id={Cursor}")
+                                ENode = Nothing
+                            Case Sections.SavedPosts
+                                SavedPostsDownload(String.Empty, Token)
+                                Exit Sub
+                            Case Sections.Tagged
+                                Dim h$ = AConvert(Of String)(MySiteSettings.HashTagged.Value, String.Empty)
+                                If h.IsEmptyString Then Throw New ExitException
                                 Dim vars$ = "{""id"":" & ID & ",""first"":50,""after"":""" & Cursor & """}"
                                 vars = SymbolsConverter.ASCII.EncodeSymbolsOnly(vars)
-                                URL = $"https://www.instagram.com/graphql/query/?query_hash={_InstaHash}&variables={vars}"
+                                URL = $"https://www.instagram.com/graphql/query/?query_hash={h}&variables={vars}"
                                 ENode = {"data", "user", 0}
-                            Case Sections.Tagged
-                                URL = $"https://i.instagram.com/api/v1/usertags/{ID}/feed/?count=50&max_id={Cursor}"
-                                ENode = {"items"}
                                 SpecFolder = TaggedFolder
                             Case Sections.Stories
                                 If Not StoriesRequested Then
@@ -303,7 +431,7 @@ Namespace API.Instagram
                                 If StoriesList.ListExists Then
                                     Continue Do
                                 Else
-                                    Throw New ExitException(_DownloadComplete)
+                                    Throw New ExitException
                                 End If
                         End Select
 
@@ -316,78 +444,73 @@ Namespace API.Instagram
                         'Parsing
                         If Not r.IsEmptyString Then
                             Using j As EContainer = JsonDocument.Parse(r).XmlIfNothing
-                                n = j.ItemF(ENode).XmlIfNothing
+                                n = If(ENode Is Nothing, j, j.ItemF(ENode)).XmlIfNothing
                                 If n.Count > 0 Then
                                     Select Case Section
-                                        Case Sections.Timeline, Sections.SavedPosts
+                                        Case Sections.Timeline
-                                            If n.Contains("page_info") Then
+                                            With n
-                                                With n("page_info")
+                                                HasNextPage = .Value("more_available").FromXML(Of Boolean)(False)
-                                                    HasNextPage = .Value("has_next_page").FromXML(Of Boolean)(False)
+                                                EndCursor = .Value("next_max_id")
-                                                    EndCursor = .Value("end_cursor")
+                                                If If(.Item("items")?.Count, 0) > 0 Then
-                                                End With
+                                                    If Not DefaultParser(.Item("items"), Section, Token) Then Throw New ExitException
-                                            End If
+                                                Else
-                                            n = n("edges").XmlIfNothing
+                                                    HasNextPage = False
-                                            If n.Count > 0 Then
+                                                End If
-                                                For Each nn In n
+                                            End With
-                                                    ThrowAny(Token)
-                                                    node = nn(0).XmlIfNothing
-                                                    If IsSavedPosts Then
-                                                        PostID = node.Value("shortcode")
-                                                        If Not PostID.IsEmptyString AndAlso _TempPostsList.Contains(PostID) Then Throw New ExitException(_DownloadComplete)
-                                                    End If
-                                                    PostID = node.Value("id")
-                                                    Pinned = CBool(If(node("pinned_for_users")?.Count, 0))
-                                                    If Not PostID.IsEmptyString And _TempPostsList.Contains(PostID) And Not Pinned Then Throw New ExitException(_DownloadComplete)
-                                                    _TempPostsList.Add(PostID)
-                                                    PostDate = node.Value("taken_at_timestamp")
-                                                    If IsSavedPosts Then
-                                                        _SavedPostsIDs.Add(PostID)
-                                                    Else
-                                                        Select Case CheckDatesLimit(PostDate, DateProvider)
-                                                            Case DateResult.Skip : Continue For
-                                                            Case DateResult.Exit : If Not Pinned Then Throw New ExitException(_DownloadComplete)
-                                                        End Select
-                                                        ObtainMedia(node, PostID, PostDate, SpecFolder)
-                                                    End If
-                                                Next
-                                            End If
                                         Case Sections.Tagged
-                                            HasNextPage = j.Value("more_available").FromXML(Of Boolean)(False)
+                                            With n
-                                            EndCursor = j.Value("next_max_id")
+                                                If .Contains("page_info") Then
-                                            For Each nn In n
+                                                    With .Item("page_info")
-                                                PostID = $"Tagged_{nn.Value("id")}"
+                                                        HasNextPage = .Value("has_next_page").FromXML(Of Boolean)(False)
-                                                If Not PostID.IsEmptyString And _TempPostsList.Contains(PostID) Then Throw New ExitException(_DownloadComplete)
+                                                        EndCursor = .Value("end_cursor")
-                                                _TempPostsList.Add(PostID)
+                                                    End With
-                                                ObtainMedia2(nn, PostID, SpecFolder)
+                                                Else
-                                                DownloadedTags += 1
+                                                    HasNextPage = False
-                                                If DownloadTagsLimit.HasValue AndAlso DownloadedTags >= DownloadTagsLimit.Value Then Throw New ExitException(_DownloadComplete)
+                                                End If
-                                            Next
+                                                If If(.Item("edges")?.Count, 0) > 0 Then
-                                            If TaggedLimitsNotifications Then
+                                                    For Each nn In .Item("edges")
-                                                TaggedCount = j.Value("total_count").FromXML(Of Integer)(0)
+                                                        PostIDKV = New PostKV(Section)
+                                                        If nn.Count > 0 AndAlso nn(0).Count > 0 Then
+                                                            With nn(0)
+                                                                PostIDKV = New PostKV(.Value("shortcode"), .Value("id"), Section)
+                                                                If PostKvExists(PostIDKV) Then
+                                                                    Throw New ExitException
+                                                                Else
+                                                                    If Not DownloadTagsLimit.HasValue OrElse PostsToReparse.Count + 1 < DownloadTagsLimit.Value Then
+                                                                        _TempPostsList.Add(GetPostIdBySection(PostIDKV.ID, Section))
+                                                                        PostsKVIDs.ListAddValue(PostIDKV, LAP.NotContainsOnly)
+                                                                        PostsToReparse.ListAddValue(PostIDKV, LNC)
+                                                                    ElseIf DownloadTagsLimit.HasValue OrElse PostsToReparse.Count + 1 >= DownloadTagsLimit.Value Then
+                                                                        Throw New ExitException
+                                                                    End If
+                                                                End If
+                                                            End With
+                                                        End If
+                                                    Next
+                                                Else
+                                                    HasNextPage = False
+                                                End If
+                                            End With
+                                            If TaggedLimitsNotifications(PostsToReparse.Count) Then
                                                 TaggedChecked = True
-                                                If TaggedLimitsNotifications(TaggedCount) AndAlso
+                                                If TaggedContinue(PostsToReparse.Count) = DialogResult.Cancel Then Throw New ExitException
-                                                   TaggedContinue(TaggedCount) = DialogResult.Cancel Then Throw New ExitException(_DownloadComplete)
                                             End If
                                     End Select
                                 Else
-                                    If j.Value("status") = "ok" AndAlso j({"data", "user"}).XmlIfNothing.Count = 0 AndAlso
+                                    If j.Value("status") = "ok" AndAlso If(j("items")?.Count, 0) = 0 AndAlso
                                        _TempMediaList.Count = 0 AndAlso Section = Sections.Timeline Then _
-                                       UserExists = False : Throw New ExitException(_DownloadComplete)
+                                       UserExists = False : Throw New ExitException
                                 End If
                             End Using
                         Else
-                            Throw New ExitException(_DownloadComplete)
+                            Throw New ExitException
                         End If
-                        _DownloadComplete = True
+                        dValue = 0
                         If HasNextPage And Not EndCursor.IsEmptyString Then DownloadData(EndCursor, Section, Token)
                     Catch eex As ExitException
                         Throw eex
-                    Catch oex As OperationCanceledException When Token.IsCancellationRequested
-                        Exit Do
-                    Catch dex As ObjectDisposedException When Disposed
-                        Exit Do
                     Catch ex As Exception
-                        If DownloadingException(ex, $"data downloading error [{URL}]", False, Section) = 1 Then Continue Do Else Exit Do
+                        dValue = ProcessException(ex, Token, $"data downloading error [{URL}]",, Section, False)
                     End Try
                 Loop
             Catch eex2 As ExitException
@@ -400,10 +523,10 @@ Namespace API.Instagram
         End Sub
         Private Sub DownloadPosts(ByVal Token As CancellationToken)
             Dim URL$ = String.Empty
-            Dim _DownloadComplete As Boolean = False
+            Dim dValue% = 1
             Dim _Index% = 0
             Try
-                Do While Not _DownloadComplete
+                Do While dValue = 1
                     ThrowAny(Token)
                     If Not Ready() Then Thread.Sleep(10000) : ThrowAny(Token) : Continue Do
                     ReconfigureAwaiter()
@@ -411,13 +534,11 @@ Namespace API.Instagram
                     Try
                         Dim r$
                         Dim j As EContainer, jj As EContainer
-                        Dim _MediaObtained As Boolean
+                        If PostsToReparse.Count > 0 And _Index <= PostsToReparse.Count - 1 Then
-                        If _SavedPostsIDs.Count > 0 And _Index <= _SavedPostsIDs.Count - 1 Then
                             Dim e As New ErrorsDescriber(EDP.ThrowException)
-                            For i% = _Index To _SavedPostsIDs.Count - 1
+                            For i% = _Index To PostsToReparse.Count - 1
                                 _Index = i
-                                'URL = $"https://instagram.com/p/{_SavedPostsIDs(i)}/?__a=1"
+                                URL = $"https://www.instagram.com/api/v1/media/{PostsToReparse(i).ID}/info/"
-                                URL = $"https://i.instagram.com/api/v1/media/{_SavedPostsIDs(i)}/info/"
                                 ThrowAny(Token)
                                 NextRequest(((i + 1) Mod 5) = 0)
                                 ThrowAny(Token)
@@ -427,17 +548,9 @@ Namespace API.Instagram
                                 If Not r.IsEmptyString Then
                                     j = JsonDocument.Parse(r)
                                     If Not j Is Nothing Then
-                                        _MediaObtained = False
+                                        If If(j("items")?.Count, 0) > 0 Then
-                                        If j.Contains({"graphql", "shortcode_media"}) Then
-                                            With j({"graphql", "shortcode_media"}).XmlIfNothing
-                                                If .Count > 0 Then ObtainMedia(.Self, _SavedPostsIDs(i), String.Empty, String.Empty) : _MediaObtained = True
-                                            End With
-                                        End If
-                                        If Not _MediaObtained AndAlso j.Contains("items") Then
                                             With j("items")
-                                                If .Count > 0 Then
+                                                For Each jj In .Self : ObtainMedia(jj, PostsToReparse(i).ID) : Next
-                                                    For Each jj In .Self : ObtainMedia2(jj, _SavedPostsIDs(i)) : Next
-                                                End If
                                             End With
                                         End If
                                         j.Dispose()
@@ -445,24 +558,82 @@ Namespace API.Instagram
                                 End If
                             Next
                         End If
-                        _DownloadComplete = True
+                        dValue = 0
                     Catch eex As ExitException
                         Throw eex
-                    Catch oex As OperationCanceledException When Token.IsCancellationRequested
-                        Exit Do
-                    Catch dex As ObjectDisposedException When Disposed
-                        Exit Do
                     Catch ex As Exception
-                        If DownloadingException(ex, $"downloading saved posts error [{URL}]", False, Sections.SavedPosts) = 1 Then Continue Do Else Exit Do
+                        dValue = ProcessException(ex, Token, $"downloading posts error [{URL}]",, Sections.Tagged, False)
                     End Try
                 Loop
             Catch eex2 As ExitException
             Catch oex2 As OperationCanceledException When Token.IsCancellationRequested Or oex2.HelpLink = InstAborted
                 If oex2.HelpLink = InstAborted Then HasError = True
             Catch DoEx As Exception
-                ProcessException(DoEx, Token, $"downloading saved posts error [{URL}]",, Sections.SavedPosts)
+                ProcessException(DoEx, Token, $"downloading posts error [{URL}]",, Sections.Tagged)
             End Try
         End Sub
+        Private Sub SavedPostsDownload(ByVal Cursor As String, ByVal Token As CancellationToken)
+            Dim URL$ = $"https://www.instagram.com/api/v1/feed/saved/posts/?max_id={Cursor}"
+            Dim HasNextPage As Boolean = False
+            Dim NextCursor$ = String.Empty
+            ThrowAny(Token)
+            Dim r$ = Responser.GetResponse(URL)
+            Dim nodes As IEnumerable(Of EContainer) = Nothing
+            If Not r.IsEmptyString Then
+                Using e As EContainer = JsonDocument.Parse(r)
+                    If If(e?.Count, 0) > 0 Then
+                        With e
+                            HasNextPage = .Value("more_available").FromXML(Of Boolean)(False)
+                            NextCursor = .Value("next_max_id")
+                            If .Contains("items") Then nodes = (From ee As EContainer In .Item("items") Where ee.Count > 0 Select ee(0))
+                        End With
+                        If nodes.ListExists Then
+                            DefaultParser(nodes, Sections.SavedPosts, Token)
+                            If HasNextPage And Not NextCursor.IsEmptyString Then SavedPostsDownload(NextCursor, Token)
+                        End If
+                    End If
+                End Using
+            End If
+        End Sub
+        Private Function DefaultParser(ByVal Items As IEnumerable(Of EContainer), ByVal Section As Sections, ByVal Token As CancellationToken,
+                                       Optional ByVal SpecFolder As String = Nothing) As Boolean
+            ThrowAny(Token)
+            If Items.Count > 0 Then
+                Dim PostIDKV As PostKV
+                Dim Pinned As Boolean
+                Dim PostDate$
+                If SpecFolder.IsEmptyString Then
+                    Select Case Section
+                        Case Sections.Tagged : SpecFolder = TaggedFolder
+                        Case Sections.Stories : SpecFolder = StoriesFolder
+                        Case Else : SpecFolder = String.Empty
+                    End Select
+                End If
+                For Each nn In Items
+                    With nn
+                        PostIDKV = New PostKV(.Value("code"), .Value("id"), Section)
+                        Pinned = .Contains("timeline_pinned_user_ids")
+                        If PostKvExists(PostIDKV) Then
+                            If Not Pinned Then Return False
+                        Else
+                            _TempPostsList.Add(PostIDKV.ID)
+                            PostsKVIDs.ListAddValue(PostIDKV, LNC)
+                            PostDate = .Value("taken_at")
+                            If Not IsSavedPosts Then
+                                Select Case CheckDatesLimit(PostDate, DateProvider)
+                                    Case DateResult.Skip : Continue For
+                                    Case DateResult.Exit : If Not Pinned Then Return False
+                                End Select
+                            End If
+                            ObtainMedia(.Self, PostIDKV.ID, SpecFolder, PostDate)
+                        End If
+                    End With
+                Next
+                Return True
+            Else
+                Return False
+            End If
+        End Function
 #End Region
 #Region "Code ID converters"
         Private Shared Function CodeToID(ByVal Code As String) As String
@@ -485,25 +656,7 @@ Namespace API.Instagram
         End Function
 #End Region
 #Region "Obtain Media"
-        Private Sub ObtainMedia(ByVal node As EContainer, ByVal PostID As String, ByVal PostDate As String, ByVal SpecFolder As String)
+        Private Sub ObtainMedia(ByVal n As EContainer, ByVal PostID As String, Optional ByVal SpecialFolder As String = Nothing,
-            Dim CreateMedia As Action(Of EContainer) =
-                Sub(ByVal e As EContainer)
-                    Dim t As UTypes = If(e.Value("is_video").FromXML(Of Boolean)(False), UTypes.Video, UTypes.Picture)
-                    Dim tmpValue$
-                    If t = UTypes.Picture Then
-                        tmpValue = e.Value("display_url")
-                    Else
-                        tmpValue = e.Value("video_url")
-                    End If
-                    If Not tmpValue.IsEmptyString Then _TempMediaList.ListAddValue(MediaFromData(t, tmpValue, PostID, PostDate, SpecFolder), LNC)
-                End Sub
-            If node.Contains({"edge_sidecar_to_children", "edges"}) Then
-                For Each edge As EContainer In node({"edge_sidecar_to_children", "edges"}) : CreateMedia(edge("node").XmlIfNothing) : Next
-            Else
-                CreateMedia(node)
-            End If
-        End Sub
-        Private Sub ObtainMedia2(ByVal n As EContainer, ByVal PostID As String, Optional ByVal SpecialFolder As String = Nothing,
                                  Optional ByVal DateObj As String = Nothing)
             Try
                 Dim img As Predicate(Of EContainer) = Function(_img) Not _img.Name.IsEmptyString AndAlso _img.Name.StartsWith("image_versions") AndAlso _img.Count > 0
@@ -575,7 +728,7 @@ Namespace API.Instagram
                                 DateObj = mDate(n)
                                 With n("carousel_media").XmlIfNothing
                                     If .Count > 0 Then
-                                        For Each d In .Self : ObtainMedia2(d, PostID, SpecialFolder, DateObj) : Next
+                                        For Each d In .Self : ObtainMedia(d, PostID, SpecialFolder, DateObj) : Next
                                     End If
                                 End With
                         End Select
@@ -588,22 +741,6 @@ Namespace API.Instagram
         End Sub
 #End Region
 #Region "GetUserId"
-        <Obsolete> Private Sub GetUserId_Old()
-            Try
-                Dim r$ = Responser.GetResponse($"https://www.instagram.com/{Name}/?__a=1",, EDP.ThrowException)
-                If Not r.IsEmptyString Then
-                    Using j As EContainer = JsonDocument.Parse(r).XmlIfNothing
-                        ID = j({"graphql", "user"}, "id").XmlIfNothingValue
-                    End Using
-                End If
-            Catch ex As Exception
-                If Responser.StatusCode = HttpStatusCode.NotFound Or Responser.StatusCode = HttpStatusCode.BadRequest Then
-                    Throw ex
-                Else
-                    LogError(ex, "get Instagram user id")
-                End If
-            End Try
-        End Sub
         Private Sub GetUserId()
             Try
                 Dim r$ = Responser.GetResponse($"https://i.instagram.com/api/v1/users/web_profile_info/?username={Name}",, EDP.ThrowException)
@@ -652,7 +789,7 @@ Namespace API.Instagram
                                                 pid = storyID & s.Value("id")
                                                 If Not _TempPostsList.Contains(pid) Then
                                                     ThrowAny(Token)
-                                                    ObtainMedia2(s, pid, sFolder)
+                                                    ObtainMedia(s, pid, sFolder)
                                                     _TempPostsList.Add(pid)
                                                 End If
                                             Next
@@ -731,8 +868,7 @@ Namespace API.Instagram
                 Dim s As Sections = DirectCast(Section, Sections)
                 Select Case s
                     Case Sections.Timeline : MySiteSettings.DownloadTimeline.Value = False
-                    Case Sections.SavedPosts : MySiteSettings.DownloadSaved.Value = False
+                    Case Else : MySiteSettings.DownloadTagged.Value = False
-                    Case Else : MySiteSettings.DownloadStoriesTagged.Value = False
                 End Select
                 MyMainLOG = $"[{s}] downloading is disabled until you update your credentials".ToUpper
             End If
@@ -750,7 +886,7 @@ Namespace API.Instagram
         End Function
 #End Region
 #Region "Standalone downloader"
-        Friend Shared Function GetVideoInfo(ByVal URL As String, ByVal r As Response) As IEnumerable(Of UserMedia)
+        Friend Shared Function GetVideoInfo(ByVal URL As String, ByVal r As Responser) As IEnumerable(Of UserMedia)
             Try
                 If Not URL.IsEmptyString AndAlso URL.Contains("instagram.com") Then
                     Dim PID$ = RegexReplace(URL, RParams.DMS(".*?instagram.com/p/([_\w\d]+)", 1))
@@ -758,9 +894,9 @@ Namespace API.Instagram
                     If Not PID.IsEmptyString Then
                         Using t As New UserData
                             t.SetEnvironment(Settings(InstagramSiteKey), Nothing, False, False)
-                            t.Responser = New Response
+                            t.Responser = New Responser
                             t.Responser.Copy(r)
-                            t._SavedPostsIDs.Add(PID)
+                            t.PostsToReparse.Add(New PostKV With {.ID = PID})
                             t.DownloadPosts(Nothing)
                             Return ListAddList(Nothing, t._TempMediaList)
                         End Using
@@ -774,7 +910,13 @@ Namespace API.Instagram
 #End Region
 #Region "IDisposable Support"
         Protected Overrides Sub Dispose(ByVal disposing As Boolean)
-            If Not disposedValue And disposing Then _SavedPostsIDs.Clear()
+            If Not disposedValue Then
+                UpdateResponser()
+                If disposing Then
+                    PostsKVIDs.Clear()
+                    PostsToReparse.Clear()
+                End If
+            End If
             MyBase.Dispose(disposing)
         End Sub
 #End Region

SCrawler/API/LPSG/UserData.vb

@@ -88,7 +88,7 @@ Namespace API.LPSG
             End If
         End Sub
         Protected Overrides Sub DownloadContent(ByVal Token As CancellationToken)
-            With Responser : .Mode = Response.Modes.WebClient : .ResetStatus() : End With
+            With Responser : .Mode = Responser.Modes.WebClient : .ResetStatus() : End With
             UseResponserClient = True
             DownloadContentDefault(Token)
         End Sub

SCrawler/API/PornHub/M3U8.vb

@@ -14,7 +14,7 @@ Namespace API.PornHub
     Friend NotInheritable Class M3U8
         Private Sub New()
         End Sub
-        Private Shared Function GetUrlsList(ByVal URL As String, ByVal Responser As Response) As List(Of String)
+        Private Shared Function GetUrlsList(ByVal URL As String, ByVal Responser As Responser) As List(Of String)
             Dim appender$ = RegexReplace(URL, Regex_M3U8_FileUrl)
             Dim r$ = Responser.GetResponse(URL)
             If Not r.IsEmptyString Then
@@ -35,7 +35,7 @@ Namespace API.PornHub
             End If
             Return Nothing
         End Function
-        Friend Shared Function Download(ByVal URL As String, ByVal Responser As Response, ByVal Destination As SFile) As SFile
+        Friend Shared Function Download(ByVal URL As String, ByVal Responser As Responser, ByVal Destination As SFile) As SFile
             Return M3U8Base.Download(GetUrlsList(URL, Responser), Destination, Responser)
         End Function
     End Class

SCrawler/API/PornHub/SiteSettings.vb

@@ -41,6 +41,7 @@ Namespace API.PornHub
         Friend Sub New()
             MyBase.New("PornHub", "pornhub.com")
             Responser.CurlPath = $"cURL\curl.exe"
+            Responser.CurlArgumentsRight = "--ssl-no-revoke"
             CurlPathExists = Responser.CurlPath.Exists
             Responser.DeclaredError = EDP.ThrowException
 
@@ -69,7 +70,7 @@ Namespace API.PornHub
         End Function
         Friend Overrides Function GetSpecialData(ByVal URL As String, ByVal Path As String, ByVal AskForPath As Boolean) As IEnumerable
             If Available(ISiteSettings.Download.Main, True) Then
-                Using resp As Response = Responser.Copy
+                Using resp As Responser = Responser.Copy
                     Dim spf$ = String.Empty
                     Dim f As SFile = GetSpecialDataFile(Path, AskForPath, spf)
                     Dim m As UserMedia = UserData.GetVideoInfo(URL, resp, f)
@@ -105,17 +106,14 @@ Namespace API.PornHub
             With DirectCast(User, UserData) : Return String.Format(UrlPatternUser, .PersonType, .NameTrue) : End With
         End Function
         Friend Overrides Function GetUserPostUrl(ByVal User As UserDataBase, ByVal Media As UserMedia) As String
-            'TODELETE: remove comment
+            Return Media.URL_BASE
-            Return Media.URL_BASE '$"https://www.pornhub.com/view_video.php?viewkey={Media.Post.ID}"
         End Function
 #End Region
 #Region "User options"
         Friend Overrides Sub UserOptions(ByRef Options As Object, ByVal OpenForm As Boolean)
-            Dim e As UserExchangeOptions = Nothing
+            If Options Is Nothing OrElse Not TypeOf Options Is UserExchangeOptions Then Options = New UserExchangeOptions(Me)
-            If Not Options Is Nothing AndAlso TypeOf Options Is UserExchangeOptions Then e = Options
-            If e Is Nothing Then e = New UserExchangeOptions(Me)
             If OpenForm Then
-                Using f As New OptionsForm(e) : f.ShowDialog() : End Using
+                Using f As New OptionsForm(Options) : f.ShowDialog() : End Using
             End If
         End Sub
 #End Region

SCrawler/API/PornHub/UserData.vb

@@ -178,7 +178,7 @@ Namespace API.PornHub
         Protected Overrides Sub DownloadDataF(ByVal Token As CancellationToken)
             Try
                 Responser.ResetStatus()
-                If PersonType = PersonTypeUser Then Responser.Mode = Response.Modes.Curl
+                If PersonType = PersonTypeUser Then Responser.Mode = Responser.Modes.Curl
 
                 If IsSavedPosts Then VideoPageModel = VideoPageModels.Favorite
 
@@ -187,7 +187,7 @@ Namespace API.PornHub
                 Dim __videoDone As Boolean = False
                 Dim d%
                 If DownloadVideos Then
-                    If PersonType = PersonTypeUser Then Responser.Mode = Response.Modes.Curl : Responser.Method = "POST"
+                    If PersonType = PersonTypeUser Then Responser.Mode = Responser.Modes.Curl : Responser.Method = "POST"
                     If VideoPageModel = VideoPageModels.Undefined Then
                         __continue = False
                         d = DownloadUserVideos(page, Token)
@@ -210,13 +210,14 @@ Namespace API.PornHub
                     If __continue And Not __videoDone Then
                         Do While DownloadUserVideos(page, Token) = DataDownloaded And page < 100 : page += 1 : Loop
                     End If
+                    If _TempMediaList.Count > 0 Then _TempMediaList.RemoveAll(Function(m) Not m.Type = UTypes.m3u8 And Not m.Type = UTypes.VideoPre)
                 End If
 
                 Responser.Method = "GET"
                 If DownloadGifs And Not IsSavedPosts Then DownloadUserGifs(Token)
                 If DownloadImages Then DownloadUserPhotos(Token)
             Finally
-                Responser.Mode = Response.Modes.Default
+                Responser.Mode = Responser.Modes.Default
                 Responser.Method = "GET"
             End Try
         End Sub
@@ -256,7 +257,7 @@ Namespace API.PornHub
                     If PersonType = PersonTypeUser And r.Contains(HtmlPageNotFoundVideo) Then Return DataDownloaded_NotFound
                     Dim l As List(Of UserVideo) = RegexFields(Of UserVideo)(r, {RegexVideo_Video_All}, {1, 2})
                     Dim lw As List(Of UserVideo) = Nothing
-                    If Not PersonType = PersonTypeUser Then RegexFields(Of UserVideo)(r, {RegexVideo_Video_Wrong}, RegexVideo_Video_Wrong_Fields)
+                    If Not PersonType = PersonTypeUser Then lw = RegexFields(Of UserVideo)(r, {RegexVideo_Video_Wrong}, RegexVideo_Video_Wrong_Fields)
                     If l.ListExists Then
                         If lw.ListExists Then l.ListWithRemove(lw)
                         If l.Count > 0 Then
@@ -346,12 +347,12 @@ Namespace API.PornHub
                     If PhotoPageModel = PhotoPageModels.Undefined Then
                         If DownloadUserPhotos_ModelHub(Token) Then PhotoPageModel = PhotoPageModels.ModelHubPage
                         ThrowAny(Token)
-                        If PhotoPageModel = PhotoPageModels.Undefined AndAlso DownloadPhotoOnlyFromModelHub AndAlso
+                        If PhotoPageModel = PhotoPageModels.Undefined AndAlso Not DownloadPhotoOnlyFromModelHub AndAlso
                            DownloadUserPhotos_PornHub(Token) Then PhotoPageModel = PhotoPageModels.PornHubPage
                     Else
                         Select Case PhotoPageModel
                             Case PhotoPageModels.ModelHubPage : DownloadUserPhotos_ModelHub(Token)
-                            Case PhotoPageModels.PornHubPage : If DownloadPhotoOnlyFromModelHub Then DownloadUserPhotos_PornHub(Token)
+                            Case PhotoPageModels.PornHubPage : If Not DownloadPhotoOnlyFromModelHub Then DownloadUserPhotos_PornHub(Token)
                         End Select
                     End If
                 ElseIf Not DownloadPhotoOnlyFromModelHub Then
@@ -359,7 +360,7 @@ Namespace API.PornHub
                 End If
                 ThrowAny(Token)
             Catch ex As Exception
-                ProcessException(ex, Token, $"photos downloading error")
+                ProcessException(ex, Token, "photos downloading error")
             End Try
         End Sub
         Private Function DownloadUserPhotos_ModelHub(ByVal Token As CancellationToken) As Boolean
@@ -624,7 +625,7 @@ Namespace API.PornHub
         End Function
 #End Region
 #Region "Standalone downloader"
-        Friend Shared Function GetVideoInfo(ByVal URL As String, ByVal Responser As Response, ByVal Destination As SFile) As UserMedia
+        Friend Shared Function GetVideoInfo(ByVal URL As String, ByVal Responser As Responser, ByVal Destination As SFile) As UserMedia
             Try
                 Dim r$ = Responser.Curl(URL)
                 If Not r.IsEmptyString Then

SCrawler/API/Reddit/UserData.vb

@@ -136,7 +136,7 @@ Namespace API.Reddit
             _CrossPosts.Clear()
             If Not IsSavedPosts AndAlso (IsChannel AndAlso Not ChannelInfo Is Nothing) Then
                 If Not Responser Is Nothing Then Responser.Dispose()
-                Responser = New Response
+                Responser = New Responser
                 Responser.Copy(MySiteSettings.Responser)
                 ChannelPostsNames.ListAddList(ChannelInfo.PostsAll.Select(Function(p) p.ID), LNC)
                 If Not ViewMode = CView.New Then ChannelPostsNames.ListAddList(ChannelInfo.PostsNames, LNC)
@@ -524,7 +524,7 @@ Namespace API.Reddit
             End Try
         End Function
         Protected Overrides Sub ReparseVideo(ByVal Token As CancellationToken)
-            Dim RedGifsResponser As Response = Nothing
+            Dim RedGifsResponser As Responser = Nothing
             Try
                 ThrowAny(Token)
                 Const v2 As UTypes = UTypes.VideoPre + UTypes.m3u8
@@ -581,7 +581,7 @@ Namespace API.Reddit
         End Sub
         Protected Overrides Sub ReparseMissing(ByVal Token As CancellationToken)
             Dim rList As New List(Of Integer)
-            Dim RedGifsResponser As Response = Nothing
+            Dim RedGifsResponser As Responser = Nothing
             Try
                 If Not ChannelInfo Is Nothing Or SaveToCache Then Exit Sub
                 If ContentMissingExists Then
@@ -658,11 +658,12 @@ Namespace API.Reddit
             Public Overrides Sub Perform(Optional ByVal Value As Double = 1)
             End Sub
         End Class
-        Friend Shared Function GetVideoInfo(ByVal URL As String, ByVal resp As Response, ByVal f As SFile, ByVal SpecialFolder As String) As IEnumerable(Of UserMedia)
+        Friend Shared Function GetVideoInfo(ByVal URL As String, ByVal resp As Responser, ByVal f As SFile, ByVal SpecialFolder As String) As IEnumerable(Of UserMedia)
             Try
                 If Not URL.IsEmptyString Then
                     Using r As New UserData
-                        r.Responser = New Response
+                        r.SetEnvironment(Settings(RedditSiteKey), Nothing, False, False)
+                        r.Responser = New Responser
                         r.Responser.Copy(resp)
                         r.ParsePost(URL)
                         If r._TempMediaList.Count > 0 Then
@@ -713,7 +714,7 @@ Namespace API.Reddit
         End Function
 #End Region
         Protected Overrides Sub DownloadContent(ByVal Token As CancellationToken)
-            Dim RedGifsResponser As Response = Nothing
+            Dim RedGifsResponser As Responser = Nothing
             Try
                 Const _RFN$ = "RedditVideo"
                 Const RFN$ = _RFN & "{0}"

SCrawler/API/Redgifs/SiteSettings.vb

@@ -9,10 +9,10 @@
 Imports SCrawler.API.Base
 Imports SCrawler.Plugin
 Imports SCrawler.Plugin.Attributes
+Imports PersonalUtilities.Forms
 Imports PersonalUtilities.Functions.XML
 Imports PersonalUtilities.Functions.RegularExpressions
 Imports PersonalUtilities.Tools.Web.Clients
-Imports PersonalUtilities.Tools.Web.Cookies
 Imports PersonalUtilities.Tools.Web.Documents.JSON
 Imports UTypes = SCrawler.API.Base.UserMedia.Types
 Imports UStates = SCrawler.API.Base.UserMedia.States
@@ -30,36 +30,53 @@ Namespace API.RedGifs
                 Return My.Resources.SiteResources.RedGifsPic_32
             End Get
         End Property
-        <PropertyOption(AllowNull:=False, ControlText:="Token", ControlToolTip:="Bearer token")>
+        <PropertyOption(ControlToolTip:="Bearer token", AllowNull:=False), ControlNumber(1)>
-        Friend Property Token As PropertyValue
+        Friend ReadOnly Property Token As PropertyValue
-        <PXML> Friend Property TokenLastDateUpdated As PropertyValue
+        <PXML> Friend ReadOnly Property TokenLastDateUpdated As PropertyValue
-        <DoNotUse> Friend ReadOnly Property NoCredentialsResponser As Response
         Private Const TokenName As String = "authorization"
+#Region "TokenUpdateInterval"
+        <PropertyOption(ControlText:="Token refresh interval", ControlToolTip:="Interval (in minutes) to refresh the token", AllowNull:=False, LeftOffset:=120),
+            PXML, ControlNumber(0)>
+        Friend ReadOnly Property TokenUpdateInterval As PropertyValue
+        Private Class TokenIntervalProvider : Implements IFieldsCheckerProvider
+            Private Property ErrorMessage As String Implements IFieldsCheckerProvider.ErrorMessage
+            Private Property Name As String Implements IFieldsCheckerProvider.Name
+            Private Property TypeError As Boolean Implements IFieldsCheckerProvider.TypeError
+            Private Function Convert(ByVal Value As Object, ByVal DestinationType As Type, ByVal Provider As IFormatProvider,
+                                     Optional ByVal NothingArg As Object = Nothing, Optional ByVal e As ErrorsDescriber = Nothing) As Object Implements ICustomProvider.Convert
+                TypeError = False
+                ErrorMessage = String.Empty
+                If Not ACheck(Of Integer)(Value) Then
+                    TypeError = True
+                ElseIf CInt(Value) > 0 Then
+                    Return Value
+                Else
+                    ErrorMessage = $"The value of [{Name}] field must be greater than or equal to 1"
+                End If
+                Return Nothing
+            End Function
+            Private Function GetFormat(ByVal FormatType As Type) As Object Implements IFormatProvider.GetFormat
+                Throw New NotImplementedException("[GetFormat] is not available in the context of [TokenIntervalProvider]")
+            End Function
+        End Class
+        <Provider(NameOf(TokenUpdateInterval), FieldsChecker:=True)>
+        Private ReadOnly Property TokenUpdateIntervalProvider As IFormatProvider
+#End Region
 #End Region
 #Region "Initializer"
         Friend Sub New()
             MyBase.New(RedGifsSite, "redgifs.com")
             Dim t$ = String.Empty
             With Responser
-                Dim b As Boolean = Not .Mode = Response.Modes.WebClient
+                Dim b As Boolean = Not .Mode = Responser.Modes.WebClient
-                .Mode = Response.Modes.WebClient
+                .Mode = Responser.Modes.WebClient
-                t = .HeadersValue(TokenName)
+                t = .Headers.Value(TokenName)
                 If b Then .SaveSettings()
             End With
-            NoCredentialsResponser = New Response($"{SettingsFolderName}\Responser_{RedGifsSite}_NC.xml") With {
-                .CookiesEncryptKey = SettingsCLS.CookieEncryptKey,
-                .CookiesDomain = "redgifs.com"
-            }
-            With NoCredentialsResponser
-                If .File.Exists Then
-                    .LoadSettings()
-                Else
-                    .Cookies = New CookieKeeper(.CookiesDomain) With {.EncryptKey = SettingsCLS.CookieEncryptKey}
-                    .SaveSettings()
-                End If
-            End With
             Token = New PropertyValue(t, GetType(String), Sub(v) UpdateResponse(v))
             TokenLastDateUpdated = New PropertyValue(Now.AddYears(-1), GetType(Date))
+            TokenUpdateInterval = New PropertyValue(60 * 12, GetType(Integer))
+            TokenUpdateIntervalProvider = New TokenIntervalProvider
             UrlPatternUser = "https://www.redgifs.com/users/{0}/"
             UserRegex = RParams.DMS("[htps:/]{7,8}.*?redgifs.com/users/([^/]+)", 1)
             ImageVideoContains = "redgifs"
@@ -67,14 +84,14 @@ Namespace API.RedGifs
 #End Region
 #Region "Response updater"
         Private Sub UpdateResponse(ByVal Value As String)
-            Responser.HeadersAdd(TokenName, Value)
+            Responser.Headers.Add(TokenName, Value)
             Responser.SaveSettings()
         End Sub
 #End Region
 #Region "Token updaters"
         Friend Function UpdateTokenIfRequired() As Boolean
             Dim d As Date? = AConvert(Of Date)(TokenLastDateUpdated.Value, AModes.Var, Nothing)
-            If Not d.HasValue OrElse d.Value < Now.AddDays(-1) Then
+            If Not d.HasValue OrElse d.Value < Now.AddMinutes(-CInt(TokenUpdateInterval.Value)) Then
                 Return UpdateToken()
             Else
                 Return True
@@ -85,7 +102,7 @@ Namespace API.RedGifs
             Try
                 Dim r$
                 Dim NewToken$ = String.Empty
-                Using resp As New Response : r = resp.GetResponse("https://api.redgifs.com/v2/auth/temporary",, EDP.ThrowException) : End Using
+                Using resp As New Responser : r = resp.GetResponse("https://api.redgifs.com/v2/auth/temporary",, EDP.ThrowException) : End Using
                 If Not r.IsEmptyString Then
                     Dim j As EContainer = JsonDocument.Parse(r)
                     If Not j Is Nothing Then
@@ -126,7 +143,7 @@ Namespace API.RedGifs
         End Function
         Friend Overrides Function GetSpecialData(ByVal URL As String, ByVal Path As String, ByVal AskForPath As Boolean) As IEnumerable
             If BaseAuthExists() Then
-                Using resp As Response = Responser.Copy
+                Using resp As Responser = Responser.Copy
                     Dim m As UserMedia = UserData.GetDataFromUrlId(URL, False, resp, Settings(RedGifsSiteKey))
                     If Not m.State = UStates.Missing And Not m.State = UserData.DataGone And (m.Type = UTypes.Picture Or m.Type = UTypes.Video) Then
                         Try

SCrawler/API/Redgifs/UserData.vb

@@ -34,21 +34,15 @@ Namespace API.RedGifs
         End Sub
 #End Region
 #Region "Download functions"
-        Private NoCredentialsResponser As Response
         Protected Overrides Sub DownloadDataF(ByVal Token As CancellationToken)
-            Try
+            DownloadData(1, Token)
-                NoCredentialsResponser = MySettings.NoCredentialsResponser.Copy
-                DownloadData(1, Token)
-            Finally
-                NoCredentialsResponser.Dispose()
-            End Try
         End Sub
         Private Overloads Sub DownloadData(ByVal Page As Integer, ByVal Token As CancellationToken)
             Dim URL$ = String.Empty
             Try
                 Dim _page As Func(Of String) = Function() If(Page = 1, String.Empty, $"&page={Page}")
                 URL = $"https://api.redgifs.com/v2/users/{Name}/search?order=recent{_page.Invoke}"
-                Dim r$ = NoCredentialsResponser.GetResponse(URL,, EDP.ThrowException)
+                Dim r$ = Responser.GetResponse(URL,, EDP.ThrowException)
                 Dim postDate$, postID$
                 Dim pTotal% = 0
                 If Not r.IsEmptyString Then
@@ -70,7 +64,7 @@ Namespace API.RedGifs
                 End If
                 If pTotal > 0 And Page < pTotal Then DownloadData(Page + 1, Token)
             Catch ex As Exception
-                ProcessException(ex, Token, $"data downloading error [{URL}]",, True)
+                ProcessException(ex, Token, $"data downloading error [{URL}]")
             End Try
         End Sub
 #End Region
@@ -166,7 +160,7 @@ Namespace API.RedGifs
                 Return String.Empty
             End If
         End Function
-        Friend Shared Function GetDataFromUrlId(ByVal Obj As String, ByVal ObjIsID As Boolean, ByVal Responser As Response,
+        Friend Shared Function GetDataFromUrlId(ByVal Obj As String, ByVal ObjIsID As Boolean, ByVal Responser As Responser,
                                                 ByVal Host As Plugin.Hosts.SettingsHost) As UserMedia
             Dim URL$ = String.Empty
             Try
@@ -239,18 +233,8 @@ Namespace API.RedGifs
 #Region "Exception"
         Protected Overrides Function DownloadingException(ByVal ex As Exception, ByVal Message As String, Optional ByVal FromPE As Boolean = False,
                                                           Optional ByVal EObj As Object = Nothing) As Integer
-            Dim IsNoCredentialsResponser As Boolean = AConvert(Of Boolean)(EObj, False)
+            Dim s As WebExceptionStatus = Responser.Client.Status
-            Dim s As WebExceptionStatus = -1
+            Dim sc As HttpStatusCode = Responser.Client.StatusCode
-            Dim sc As HttpStatusCode = -1
-            If IsNoCredentialsResponser Then
-                If Not NoCredentialsResponser Is Nothing Then
-                    s = NoCredentialsResponser.Status
-                    sc = NoCredentialsResponser.StatusCode
-                End If
-            Else
-                s = Responser.Client.Status
-                sc = Responser.Client.StatusCode
-            End If
             If sc = HttpStatusCode.NotFound Or s = DataGone Then
                 UserExists = False
             ElseIf sc = HttpStatusCode.Unauthorized Then

SCrawler/API/TikTok/UserData.vb

@@ -52,7 +52,7 @@ Namespace API.TikTok
         Protected Overrides Sub DownloadContent(ByVal Token As CancellationToken)
             DownloadContentDefault(Token)
         End Sub
-        Friend Shared Function GetVideoInfo(ByVal URL As String, ByVal Responser As Response, Optional ByVal e As ErrorsDescriber = Nothing) As IEnumerable(Of UserMedia)
+        Friend Shared Function GetVideoInfo(ByVal URL As String, ByVal Responser As Responser, Optional ByVal e As ErrorsDescriber = Nothing) As IEnumerable(Of UserMedia)
             Try
                 If Not URL.IsEmptyString Then
                     Dim PostId$ = String.Empty
@@ -61,7 +61,7 @@ Namespace API.TikTok
                     Dim r$
                     PostId = RegexEnvir.ExtractPostID(URL)
                     If Not PostId.IsEmptyString Then
-                        Using resp As Response = Responser.Copy() : r = resp.GetResponse(URL,, EDP.ThrowException) : End Using
+                        Using resp As Responser = Responser.Copy() : r = resp.GetResponse(URL,, EDP.ThrowException) : End Using
                         If Not r.IsEmptyString Then
                             If RegexEnvir.GetVideoData(r, PostId, PostURL, PostDate) Then Return {MediaFromData(PostURL, PostId, PostDate)}
                         End If

SCrawler/API/Twitter/Declarations.vb

@@ -12,6 +12,7 @@ Imports PersonalUtilities.Functions.RegularExpressions
 Namespace API.Twitter
     Friend Module Declarations
         Friend Const TwitterSite As String = "Twitter"
+        Friend Const TwitterSiteKey As String = "AndyProgram_Twitter"
         Friend ReadOnly DateProvider As ADateTime = GetDateProvider()
         Friend ReadOnly VideoNode As NodeParams() = {New NodeParams("video_info", True, True, True, True, 10)}
         Friend ReadOnly VideoSizeRegEx As RParams = RParams.DMS("\d+x(\d+)", 1, EDP.ReturnValue)

SCrawler/API/Twitter/EditorExchangeOptions.vb

@@ -0,0 +1,27 @@
+' Copyright (C) 2023  Andy https://github.com/AAndyProgram
+' This program is free software: you can redistribute it and/or modify
+' it under the terms of the GNU General Public License as published by
+' the Free Software Foundation, either version 3 of the License, or
+' (at your option) any later version.
+'
+' This program is distributed in the hope that it will be useful,
+' but WITHOUT ANY WARRANTY
+Namespace API.Twitter
+    Friend Class EditorExchangeOptions
+        Friend Property GifsDownload As Boolean
+        Friend Property GifsSpecialFolder As String
+        Friend Property GifsPrefix As String
+        Friend Sub New()
+        End Sub
+        Friend Sub New(ByVal s As SiteSettings)
+            GifsDownload = s.GifsDownload.Value
+            GifsSpecialFolder = s.GifsSpecialFolder.Value
+            GifsPrefix = s.GifsPrefix.Value
+        End Sub
+        Friend Sub New(ByVal u As UserData)
+            GifsDownload = u.GifsDownload
+            GifsSpecialFolder = u.GifsSpecialFolder
+            GifsPrefix = u.GifsPrefix
+        End Sub
+    End Class
+End Namespace

SCrawler/API/Twitter/OptionsForm.Designer.vb

@@ -0,0 +1,148 @@
+' Copyright (C) 2023  Andy https://github.com/AAndyProgram
+' This program is free software: you can redistribute it and/or modify
+' it under the terms of the GNU General Public License as published by
+' the Free Software Foundation, either version 3 of the License, or
+' (at your option) any later version.
+'
+' This program is distributed in the hope that it will be useful,
+' but WITHOUT ANY WARRANTY
+Namespace API.Twitter
+    <Global.Microsoft.VisualBasic.CompilerServices.DesignerGenerated()>
+    Partial Friend Class OptionsForm : Inherits System.Windows.Forms.Form
+        <System.Diagnostics.DebuggerNonUserCode()>
+        Protected Overrides Sub Dispose(ByVal disposing As Boolean)
+            Try
+                If disposing AndAlso components IsNot Nothing Then
+                    components.Dispose()
+                End If
+            Finally
+                MyBase.Dispose(disposing)
+            End Try
+        End Sub
+        Private components As System.ComponentModel.IContainer
+        <System.Diagnostics.DebuggerStepThrough()>
+        Private Sub InitializeComponent()
+            Dim CONTAINER_MAIN As System.Windows.Forms.ToolStripContainer
+            Dim TP_MAIN As System.Windows.Forms.TableLayoutPanel
+            Dim ActionButton1 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
+            Dim resources As System.ComponentModel.ComponentResourceManager = New System.ComponentModel.ComponentResourceManager(GetType(OptionsForm))
+            Dim ActionButton2 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton()
+            Me.CH_DOWN_GIFS = New System.Windows.Forms.CheckBox()
+            Me.TXT_GIF_FOLDER = New PersonalUtilities.Forms.Controls.TextBoxExtended()
+            Me.TXT_GIF_PREFIX = New PersonalUtilities.Forms.Controls.TextBoxExtended()
+            CONTAINER_MAIN = New System.Windows.Forms.ToolStripContainer()
+            TP_MAIN = New System.Windows.Forms.TableLayoutPanel()
+            CONTAINER_MAIN.ContentPanel.SuspendLayout()
+            CONTAINER_MAIN.SuspendLayout()
+            TP_MAIN.SuspendLayout()
+            CType(Me.TXT_GIF_FOLDER, System.ComponentModel.ISupportInitialize).BeginInit()
+            CType(Me.TXT_GIF_PREFIX, System.ComponentModel.ISupportInitialize).BeginInit()
+            Me.SuspendLayout()
+            '
+            'CONTAINER_MAIN
+            '
+            '
+            'CONTAINER_MAIN.ContentPanel
+            '
+            CONTAINER_MAIN.ContentPanel.Controls.Add(TP_MAIN)
+            CONTAINER_MAIN.ContentPanel.Size = New System.Drawing.Size(304, 84)
+            CONTAINER_MAIN.Dock = System.Windows.Forms.DockStyle.Fill
+            CONTAINER_MAIN.LeftToolStripPanelVisible = False
+            CONTAINER_MAIN.Location = New System.Drawing.Point(0, 0)
+            CONTAINER_MAIN.Name = "CONTAINER_MAIN"
+            CONTAINER_MAIN.RightToolStripPanelVisible = False
+            CONTAINER_MAIN.Size = New System.Drawing.Size(304, 109)
+            CONTAINER_MAIN.TabIndex = 0
+            CONTAINER_MAIN.TopToolStripPanelVisible = False
+            '
+            'TP_MAIN
+            '
+            TP_MAIN.CellBorderStyle = System.Windows.Forms.TableLayoutPanelCellBorderStyle.[Single]
+            TP_MAIN.ColumnCount = 1
+            TP_MAIN.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Percent, 100.0!))
+            TP_MAIN.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Absolute, 20.0!))
+            TP_MAIN.Controls.Add(Me.CH_DOWN_GIFS, 0, 0)
+            TP_MAIN.Controls.Add(Me.TXT_GIF_FOLDER, 0, 1)
+            TP_MAIN.Controls.Add(Me.TXT_GIF_PREFIX, 0, 2)
+            TP_MAIN.Dock = System.Windows.Forms.DockStyle.Fill
+            TP_MAIN.Location = New System.Drawing.Point(0, 0)
+            TP_MAIN.Name = "TP_MAIN"
+            TP_MAIN.RowCount = 4
+            TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!))
+            TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 28.0!))
+            TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 28.0!))
+            TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Percent, 100.0!))
+            TP_MAIN.Size = New System.Drawing.Size(304, 84)
+            TP_MAIN.TabIndex = 0
+            '
+            'CH_DOWN_GIFS
+            '
+            Me.CH_DOWN_GIFS.AutoSize = True
+            Me.CH_DOWN_GIFS.Dock = System.Windows.Forms.DockStyle.Fill
+            Me.CH_DOWN_GIFS.Location = New System.Drawing.Point(4, 4)
+            Me.CH_DOWN_GIFS.Name = "CH_DOWN_GIFS"
+            Me.CH_DOWN_GIFS.Padding = New System.Windows.Forms.Padding(100, 0, 0, 0)
+            Me.CH_DOWN_GIFS.Size = New System.Drawing.Size(296, 19)
+            Me.CH_DOWN_GIFS.TabIndex = 0
+            Me.CH_DOWN_GIFS.Text = "Download GIFs"
+            Me.CH_DOWN_GIFS.UseVisualStyleBackColor = True
+            '
+            'TXT_GIF_FOLDER
+            '
+            ActionButton1.BackgroundImage = CType(resources.GetObject("ActionButton1.BackgroundImage"), System.Drawing.Image)
+            ActionButton1.Name = "Clear"
+            ActionButton1.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
+            Me.TXT_GIF_FOLDER.Buttons.Add(ActionButton1)
+            Me.TXT_GIF_FOLDER.CaptionText = "GIFs special folder"
+            Me.TXT_GIF_FOLDER.CaptionToolTipText = "Put the GIFs in a special folder"
+            Me.TXT_GIF_FOLDER.Dock = System.Windows.Forms.DockStyle.Fill
+            Me.TXT_GIF_FOLDER.Location = New System.Drawing.Point(4, 30)
+            Me.TXT_GIF_FOLDER.Name = "TXT_GIF_FOLDER"
+            Me.TXT_GIF_FOLDER.Size = New System.Drawing.Size(296, 22)
+            Me.TXT_GIF_FOLDER.TabIndex = 1
+            '
+            'TXT_GIF_PREFIX
+            '
+            ActionButton2.BackgroundImage = CType(resources.GetObject("ActionButton2.BackgroundImage"), System.Drawing.Image)
+            ActionButton2.Name = "Clear"
+            ActionButton2.Tag = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons.Clear
+            Me.TXT_GIF_PREFIX.Buttons.Add(ActionButton2)
+            Me.TXT_GIF_PREFIX.CaptionText = "GIF prefix"
+            Me.TXT_GIF_PREFIX.CaptionToolTipText = "This prefix will be added to the beginning of the filename"
+            Me.TXT_GIF_PREFIX.Dock = System.Windows.Forms.DockStyle.Fill
+            Me.TXT_GIF_PREFIX.Location = New System.Drawing.Point(4, 59)
+            Me.TXT_GIF_PREFIX.Name = "TXT_GIF_PREFIX"
+            Me.TXT_GIF_PREFIX.Size = New System.Drawing.Size(296, 22)
+            Me.TXT_GIF_PREFIX.TabIndex = 2
+            '
+            'OptionsForm
+            '
+            Me.AutoScaleDimensions = New System.Drawing.SizeF(6.0!, 13.0!)
+            Me.AutoScaleMode = System.Windows.Forms.AutoScaleMode.Font
+            Me.ClientSize = New System.Drawing.Size(304, 109)
+            Me.Controls.Add(CONTAINER_MAIN)
+            Me.FormBorderStyle = System.Windows.Forms.FormBorderStyle.FixedSingle
+            Me.Icon = Global.SCrawler.My.Resources.SiteResources.TwitterIcon_32
+            Me.MaximizeBox = False
+            Me.MaximumSize = New System.Drawing.Size(320, 148)
+            Me.MinimizeBox = False
+            Me.MinimumSize = New System.Drawing.Size(320, 148)
+            Me.Name = "OptionsForm"
+            Me.ShowInTaskbar = False
+            Me.SizeGripStyle = System.Windows.Forms.SizeGripStyle.Hide
+            Me.Text = "Options"
+            CONTAINER_MAIN.ContentPanel.ResumeLayout(False)
+            CONTAINER_MAIN.ResumeLayout(False)
+            CONTAINER_MAIN.PerformLayout()
+            TP_MAIN.ResumeLayout(False)
+            TP_MAIN.PerformLayout()
+            CType(Me.TXT_GIF_FOLDER, System.ComponentModel.ISupportInitialize).EndInit()
+            CType(Me.TXT_GIF_PREFIX, System.ComponentModel.ISupportInitialize).EndInit()
+            Me.ResumeLayout(False)
+
+        End Sub
+        Private WithEvents CH_DOWN_GIFS As CheckBox
+        Private WithEvents TXT_GIF_FOLDER As PersonalUtilities.Forms.Controls.TextBoxExtended
+        Private WithEvents TXT_GIF_PREFIX As PersonalUtilities.Forms.Controls.TextBoxExtended
+    End Class
+End Namespace

SCrawler/API/Twitter/OptionsForm.resx

@@ -123,4 +123,21 @@
   <metadata name="TP_MAIN.GenerateMember" type="System.Boolean, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">
     <value>False</value>
   </metadata>
+  <assembly alias="System.Drawing" name="System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
+  <data name="ActionButton1.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+    <value>
+        iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
+        xAAADsQBlSsOGwAAAIZJREFUOE+1j10KwCAMgz2b755xl/IsvnaL2K20UfbDAmEako+ZROSTafjE12Go
+        tbbB43rK5xSAQq1VYFtmeQBoqZTSreVZvgTknM8yyyjA/qodsDF9gspD2Bj6B+DH+NqzhQQAG+POMnSX
+        AFuc5QFgn6ClHh5iOQVAKNixyucB8NY0vG9JOzzyhrdq5IRgAAAAAElFTkSuQmCC
+</value>
+  </data>
+  <data name="ActionButton2.BackgroundImage" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
+    <value>
+        iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABGdBTUEAALGPC/xhBQAAAAlwSFlzAAAO
+        xAAADsQBlSsOGwAAAIZJREFUOE+1j10KwCAMgz2b755xl/IsvnaL2K20UfbDAmEako+ZROSTafjE12Go
+        tbbB43rK5xSAQq1VYFtmeQBoqZTSreVZvgTknM8yyyjA/qodsDF9gspD2Bj6B+DH+NqzhQQAG+POMnSX
+        AFuc5QFgn6ClHh5iOQVAKNixyucB8NY0vG9JOzzyhrdq5IRgAAAAAElFTkSuQmCC
+</value>
+  </data>
 </root>

SCrawler/API/Twitter/OptionsForm.vb

@@ -0,0 +1,78 @@
+' Copyright (C) 2023  Andy https://github.com/AAndyProgram
+' This program is free software: you can redistribute it and/or modify
+' it under the terms of the GNU General Public License as published by
+' the Free Software Foundation, either version 3 of the License, or
+' (at your option) any later version.
+'
+' This program is distributed in the hope that it will be useful,
+' but WITHOUT ANY WARRANTY
+Imports System.Reflection
+Imports SCrawler.Plugin.Attributes
+Imports PersonalUtilities.Forms
+Imports PersonalUtilities.Forms.Controls
+Namespace API.Twitter
+    Friend Class OptionsForm
+        Private WithEvents MyDefs As DefaultFormOptions
+        Private ReadOnly Property MyExchangeOptions As EditorExchangeOptions
+        Private ReadOnly MyGifTextProvider As SiteSettings.GifStringProvider
+        Friend Sub New(ByRef ExchangeOptions As EditorExchangeOptions)
+            InitializeComponent()
+            MyExchangeOptions = ExchangeOptions
+            MyGifTextProvider = New SiteSettings.GifStringProvider
+            MyDefs = New DefaultFormOptions(Me, Settings.Design)
+        End Sub
+        Private Sub OptionsForm_Load(sender As Object, e As EventArgs) Handles Me.Load
+            With MyDefs
+                .MyViewInitialize(True)
+                .AddOkCancelToolbar()
+                With MyExchangeOptions
+                    CH_DOWN_GIFS.Checked = .GifsDownload
+                    TXT_GIF_FOLDER.Text = .GifsSpecialFolder
+                    TXT_GIF_FOLDER.Tag = NameOf(SiteSettings.GifsSpecialFolder)
+                    TXT_GIF_PREFIX.Text = .GifsPrefix
+                    TXT_GIF_PREFIX.Tag = NameOf(SiteSettings.GifsPrefix)
+
+                    Try
+                        Dim p As PropertyOption
+                        With Settings(TwitterSiteKey)
+                            p = .PropList.Find(Function(pp) pp.Name = TXT_GIF_FOLDER.Tag).Options
+                            If Not p Is Nothing Then
+                                TXT_GIF_FOLDER.CaptionText = p.ControlText
+                                TXT_GIF_FOLDER.CaptionToolTipText = p.ControlToolTip
+                                TXT_GIF_FOLDER.CaptionToolTipEnabled = Not TXT_GIF_FOLDER.CaptionToolTipText.IsEmptyString
+                            End If
+
+                            p = .PropList.Find(Function(pp) pp.Name = TXT_GIF_PREFIX.Tag).Options
+                            If Not p Is Nothing Then
+                                TXT_GIF_PREFIX.CaptionText = p.ControlText
+                                TXT_GIF_PREFIX.CaptionToolTipText = p.ControlToolTip
+                                TXT_GIF_PREFIX.CaptionToolTipEnabled = Not TXT_GIF_PREFIX.CaptionToolTipText.IsEmptyString
+                            End If
+                        End With
+                    Catch
+                    End Try
+                End With
+                .EndLoaderOperations()
+            End With
+        End Sub
+        Private Sub MyDefs_ButtonOkClick(ByVal Sender As Object, ByVal e As KeyHandleEventArgs) Handles MyDefs.ButtonOkClick
+            With MyExchangeOptions
+                .GifsDownload = CH_DOWN_GIFS.Checked
+                .GifsSpecialFolder = TXT_GIF_FOLDER.Text
+                .GifsPrefix = TXT_GIF_PREFIX.Text
+            End With
+            MyDefs.CloseForm()
+        End Sub
+        Private Sub TXT_ActionOnTextChanged(ByVal Sender As TextBoxExtended, ByVal e As EventArgs) Handles TXT_GIF_FOLDER.ActionOnTextChanged,
+                                                                                                           TXT_GIF_PREFIX.ActionOnTextChanged
+            If Not MyDefs.Initializing Then
+                With Sender
+                    MyGifTextProvider.PropertyName = .Tag
+                    Dim s% = .SelectionStart
+                    Dim t$ = AConvert(Of String)(.Text, String.Empty, MyGifTextProvider)
+                    If Not .Text = t Then .Text = t : .Select(s, 0)
+                End With
+            End If
+        End Sub
+    End Class
+End Namespace

SCrawler/API/Twitter/SiteSettings.vb

@@ -13,10 +13,11 @@ Imports PersonalUtilities.Functions.RegularExpressions
 Imports PersonalUtilities.Tools.Web.Clients
 Imports PersonalUtilities.Tools.Web.Cookies
 Namespace API.Twitter
-    <Manifest("AndyProgram_Twitter"), SavedPosts>
+    <Manifest(TwitterSiteKey), SavedPosts, SpecialForm(False)>
     Friend Class SiteSettings : Inherits SiteSettingsBase
         Friend Const Header_Authorization As String = "authorization"
         Friend Const Header_Token As String = "x-csrf-token"
+#Region "Declarations"
         Friend Overrides ReadOnly Property Icon As Icon
             Get
                 Return My.Resources.SiteResources.TwitterIcon_32
@@ -27,17 +28,53 @@ Namespace API.Twitter
                 Return My.Resources.SiteResources.TwitterPic_400
             End Get
         End Property
-        <PropertyOption(AllowNull:=False, ControlText:="Authorization",
+#Region "Auth"
+        <PropertyOption(AllowNull:=False, IsAuth:=True, ControlText:="Authorization",
                         ControlToolTip:="Set authorization from [authorization] response header. This field must start from [Bearer] key word")>
         Private ReadOnly Property Auth As PropertyValue
-        <PropertyOption(AllowNull:=False, ControlText:="Token", ControlToolTip:="Set token from [x-csrf-token] response header")>
+        <PropertyOption(AllowNull:=False, IsAuth:=True, ControlText:="Token", ControlToolTip:="Set token from [x-csrf-token] response header")>
         Private ReadOnly Property Token As PropertyValue
-        <PropertyOption(ControlText:="Saved posts user", ControlToolTip:="Personal profile username"), PXML>
+        <PropertyOption(IsAuth:=True, ControlText:="Saved posts user", ControlToolTip:="Personal profile username"), PXML>
         Friend ReadOnly Property SavedPostsUserName As PropertyValue
-        Friend Overrides ReadOnly Property Responser As Response
+#End Region
+#Region "Other properties"
+        <PropertyOption(IsAuth:=False, ControlText:="Download GIFs"), PXML>
+        Friend ReadOnly Property GifsDownload As PropertyValue
+        <PropertyOption(IsAuth:=False, ControlText:="GIFs special folder",
+                        ControlToolTip:="Put the GIFs in a special folder" & vbCr &
+                        "This is a folder name, not an absolute path." & vbCr &
+                        "This folder(s) will be created relative to the user's root folder." & vbCr &
+                        "Examples:" & vbCr & "SomeFolderName" & vbCr & "SomeFolderName\SomeFolderName2"), PXML>
+        Friend ReadOnly Property GifsSpecialFolder As PropertyValue
+        <PropertyOption(IsAuth:=False, ControlText:="GIF prefix", ControlToolTip:="This prefix will be added to the beginning of the filename"), PXML>
+        Friend ReadOnly Property GifsPrefix As PropertyValue
+        <Provider(NameOf(GifsSpecialFolder), Interaction:=True), Provider(NameOf(GifsPrefix), Interaction:=True)>
+        Private ReadOnly Property GifStringChecker As IFormatProvider
+        Friend Class GifStringProvider : Implements ICustomProvider, IPropertyProvider
+            Friend Property PropertyName As String Implements IPropertyProvider.PropertyName
+            Private Function Convert(ByVal Value As Object, ByVal DestinationType As Type, ByVal Provider As IFormatProvider,
+                                     Optional ByVal NothingArg As Object = Nothing, Optional ByVal e As ErrorsDescriber = Nothing) As Object Implements ICustomProvider.Convert
+                Dim v$ = AConvert(Of String)(Value, String.Empty)
+                If Not v.IsEmptyString Then
+                    If PropertyName = NameOf(GifsPrefix) Then
+                        v = v.StringRemoveWinForbiddenSymbols
+                    Else
+                        v = v.StringReplaceSymbols(GetWinForbiddenSymbols.ToList.ListWithRemove("\").ToArray, String.Empty, EDP.ReturnValue)
+                        v = v.StringTrim("\")
+                    End If
+                End If
+                Return v
+            End Function
+            Private Function GetFormat(ByVal FormatType As Type) As Object Implements IFormatProvider.GetFormat
+                Throw New NotImplementedException("[GetFormat] is not available in the context of [TimersChecker]")
+            End Function
+        End Class
+#End Region
+        Friend Overrides ReadOnly Property Responser As Responser
+#End Region
         Friend Sub New()
             MyBase.New(TwitterSite)
-            Responser = New Response($"{SettingsFolderName}\Responser_{Site}.xml")
+            Responser = New Responser($"{SettingsFolderName}\Responser_{Site}.xml")
 
             Dim a$ = String.Empty
             Dim t$ = String.Empty
@@ -46,8 +83,8 @@ Namespace API.Twitter
                 If .File.Exists Then
                     If EncryptCookies.CookiesEncrypted Then .CookiesEncryptKey = SettingsCLS.CookieEncryptKey
                     .LoadSettings()
-                    a = .HeadersValue(Header_Authorization)
+                    a = .Headers.Value(Header_Authorization)
-                    t = .HeadersValue(Header_Token)
+                    t = .Headers.Value(Header_Token)
                 Else
                     .ContentType = "application/json"
                     .Accept = "*/*"
@@ -55,15 +92,15 @@ Namespace API.Twitter
                     .Cookies = New CookieKeeper(.CookiesDomain) With {.EncryptKey = SettingsCLS.CookieEncryptKey}
                     .CookiesEncryptKey = SettingsCLS.CookieEncryptKey
                     .Decoders.Add(SymbolsConverter.Converters.Unicode)
-                    .HeadersAdd("sec-ch-ua", " Not;A Brand"";v=""99"", ""Google Chrome"";v=""91"", ""Chromium"";v=""91""")
+                    .Headers.Add("sec-ch-ua", " Not;A Brand"";v=""99"", ""Google Chrome"";v=""91"", ""Chromium"";v=""91""")
-                    .HeadersAdd("sec-ch-ua-mobile", "?0")
+                    .Headers.Add("sec-ch-ua-mobile", "?0")
-                    .HeadersAdd("sec-fetch-dest", "empty")
+                    .Headers.Add("sec-fetch-dest", "empty")
-                    .HeadersAdd("sec-fetch-mode", "cors")
+                    .Headers.Add("sec-fetch-mode", "cors")
-                    .HeadersAdd("sec-fetch-site", "same-origin")
+                    .Headers.Add("sec-fetch-site", "same-origin")
-                    .HeadersAdd(Header_Token, String.Empty)
+                    .Headers.Add(Header_Token, String.Empty)
-                    .HeadersAdd("x-twitter-active-user", "yes")
+                    .Headers.Add("x-twitter-active-user", "yes")
-                    .HeadersAdd("x-twitter-auth-type", "OAuth2Session")
+                    .Headers.Add("x-twitter-auth-type", "OAuth2Session")
-                    .HeadersAdd(Header_Authorization, String.Empty)
+                    .Headers.Add(Header_Authorization, String.Empty)
                     .SaveSettings()
                 End If
             End With
@@ -72,6 +109,11 @@ Namespace API.Twitter
             Token = New PropertyValue(t, GetType(String), Sub(v) ChangeResponserFields(NameOf(Token), v))
             SavedPostsUserName = New PropertyValue(String.Empty, GetType(String))
 
+            GifsDownload = New PropertyValue(True)
+            GifsSpecialFolder = New PropertyValue(String.Empty, GetType(String))
+            GifsPrefix = New PropertyValue("GIF_")
+            GifStringChecker = New GifStringProvider
+
             UserRegex = RParams.DMS("[htps:/]{7,8}.*?twitter.com/([^/]+)", 1)
             UrlPatternUser = "https://twitter.com/{0}"
             ImageVideoContains = "twitter"
@@ -84,8 +126,8 @@ Namespace API.Twitter
                     Case NameOf(Token) : f = Header_Token
                 End Select
                 If Not f.IsEmptyString Then
-                    Responser.HeadersRemove(f)
+                    Responser.Headers.Remove(f)
-                    If Not CStr(Value).IsEmptyString Then Responser.HeadersAdd(f, CStr(Value))
+                    If Not CStr(Value).IsEmptyString Then Responser.Headers.Add(f, CStr(Value))
                     Responser.SaveSettings()
                 End If
             End If
@@ -106,5 +148,11 @@ Namespace API.Twitter
         Friend Overrides Function BaseAuthExists() As Boolean
             Return If(Responser.Cookies?.Count, 0) > 0 And ACheck(Token.Value) And ACheck(Auth.Value)
         End Function
+        Friend Overrides Sub UserOptions(ByRef Options As Object, ByVal OpenForm As Boolean)
+            If Options Is Nothing OrElse Not TypeOf Options Is EditorExchangeOptions Then Options = New EditorExchangeOptions(Me)
+            If OpenForm Then
+                Using f As New OptionsForm(Options) : f.ShowDialog() : End Using
+            End If
+        End Sub
     End Class
 End Namespace

SCrawler/API/Twitter/UserData.vb

@@ -17,15 +17,50 @@ Imports UStates = SCrawler.API.Base.UserMedia.States
 Namespace API.Twitter
     Friend Class UserData : Inherits UserDataBase
         Private Const SinglePostUrl As String = "https://api.twitter.com/1.1/statuses/show.json?id={0}&tweet_mode=extended"
+#Region "XML names"
+        Private Const Name_GifsDownload As String = "GifsDownload"
+        Private Const Name_GifsSpecialFolder As String = "GifsSpecialFolder"
+        Private Const Name_GifsPrefix As String = "GifsPrefix"
+#End Region
 #Region "Declarations"
+        Friend Property GifsDownload As Boolean
+        Friend Property GifsSpecialFolder As String
+        Friend Property GifsPrefix As String
         Private ReadOnly _DataNames As List(Of String)
-        Protected Overrides Sub LoadUserInformation_OptionalFields(ByRef Container As XmlFile, ByVal Loading As Boolean)
+#End Region
+#Region "Exchange options"
+        Friend Overrides Function ExchangeOptionsGet() As Object
+            Return New EditorExchangeOptions(Me)
+        End Function
+        Friend Overrides Sub ExchangeOptionsSet(ByVal Obj As Object)
+            If Not Obj Is Nothing AndAlso TypeOf Obj Is EditorExchangeOptions Then
+                With DirectCast(Obj, EditorExchangeOptions)
+                    GifsDownload = .GifsDownload
+                    GifsSpecialFolder = .GifsSpecialFolder
+                    GifsPrefix = .GifsPrefix
+                End With
+            End If
         End Sub
 #End Region
-#Region "Initializer"
+#Region "Initializer, loader"
         Friend Sub New()
             _DataNames = New List(Of String)
         End Sub
+        Protected Overrides Sub LoadUserInformation_OptionalFields(ByRef Container As XmlFile, ByVal Loading As Boolean)
+            If Loading Then
+                GifsDownload = Container.Value(Name_GifsDownload).FromXML(Of Boolean)(True)
+                GifsSpecialFolder = Container.Value(Name_GifsSpecialFolder)
+                If Not Container.Contains(Name_GifsPrefix) Then
+                    GifsPrefix = "GIF_"
+                Else
+                    GifsPrefix = Container.Value(Name_GifsPrefix)
+                End If
+            Else
+                Container.Add(Name_GifsDownload, GifsDownload.BoolToInteger)
+                Container.Add(Name_GifsSpecialFolder, GifsSpecialFolder)
+                Container.Add(Name_GifsPrefix, GifsPrefix)
+            End If
+        End Sub
 #End Region
 #Region "Download functions"
         Protected Overrides Sub DownloadDataF(ByVal Token As CancellationToken)
@@ -182,10 +217,11 @@ Namespace API.Twitter
                                     url = .Value("url")
                                     ff = UrlFile(url)
                                     If Not ff.IsEmptyString Then
-                                        If Not _DataNames.Contains(ff) Then
+                                        If GifsDownload And Not _DataNames.Contains(ff) Then
                                             m = MediaFromData(url, PostID, PostDate,, State)
                                             f = m.File
-                                            If Not f.IsEmptyString Then f.Name = $"GIF_{f.Name}" : m.File = f
+                                            If Not f.IsEmptyString And Not GifsPrefix.IsEmptyString Then f.Name = $"{GifsPrefix}{f.Name}" : m.File = f
+                                            If Not GifsSpecialFolder.IsEmptyString Then m.SpecialFolder = $"{GifsSpecialFolder}*"
                                             _TempMediaList.ListAddValue(m, LNC)
                                         End If
                                         Return True
@@ -263,13 +299,13 @@ Namespace API.Twitter
         End Sub
 #End Region
 #Region "Get video static"
-        Friend Shared Function GetVideoInfo(ByVal URL As String, ByVal resp As Response) As IEnumerable(Of UserMedia)
+        Friend Shared Function GetVideoInfo(ByVal URL As String, ByVal resp As Responser) As IEnumerable(Of UserMedia)
             Try
                 If URL.Contains("twitter") Then
                     Dim PostID$ = RegexReplace(URL, RParams.DM("(?<=/)\d+", 0))
                     If Not PostID.IsEmptyString Then
                         Dim r$
-                        Using rc As Response = resp.Copy() : r = rc.GetResponse(String.Format(SinglePostUrl, PostID),, EDP.ReturnValue) : End Using
+                        Using rc As Responser = resp.Copy() : r = rc.GetResponse(String.Format(SinglePostUrl, PostID),, EDP.ReturnValue) : End Using
                         If Not r.IsEmptyString Then
                             Using j As EContainer = JsonDocument.Parse(r)
                                 If j.ListExists Then

SCrawler/API/UserDataBind.vb

@@ -98,7 +98,7 @@ Namespace API
                     If Not img Is Nothing Then Return img
                 Next
             End If
-            Return GetNullPicture(Settings.MaxLargeImageHeight)
+            Return GetNullPicture(If(Settings.ViewMode.Value = ViewModes.IconLarge, Settings.MaxLargeImageHeight, Settings.MaxSmallImageHeight))
         End Function
 #End Region
         Friend Overrides ReadOnly Property DownloadedTotal(Optional ByVal Total As Boolean = True) As Integer
@@ -556,7 +556,11 @@ Namespace API
                                        "Cancel"}, vbExclamation)
                 Dim v%
                 If CollectionValue >= 0 Then
-                    v = CollectionValue
+                    Select Case CollectionValue
+                        Case 2 : v = 0
+                        Case 3 : v = 1
+                        Case Else : v = MsgBoxE(m)
+                    End Select
                 ElseIf Multiple Then
                     v = 0
                 Else

SCrawler/API/XVIDEOS/SiteSettings.vb

@@ -57,6 +57,7 @@ Namespace API.XVIDEOS
             SiteDomains = New PropertyValue(DomainsDefault, GetType(String), Sub(s) UpdateDomains())
             DownloadUHD = New PropertyValue(False)
             SavedVideosPlaylist = New PropertyValue(String.Empty, GetType(String))
+            UrlPatternUser = "https://xvideos.com/{0}"
         End Sub
         Friend Overrides Sub EndInit()
             Initialized = True
@@ -103,11 +104,14 @@ Namespace API.XVIDEOS
         End Function
 #End Region
 #Region "User: get, check"
-        Friend Overrides Function GetUserUrl(ByVal User As IPluginContentProvider, ByVal Channel As Boolean) As String
+        Friend Function GetUserUrlPart(ByVal User As UserData) As String
             Dim __user$ = User.Name.Split("_").FirstOrDefault
-            __user &= $"/{User.Name.Replace($"{User}_", String.Empty)}"
+            __user &= $"/{User.Name.Replace($"{__user}_", String.Empty)}"
             Return __user
         End Function
+        Friend Overrides Function GetUserUrl(ByVal User As IPluginContentProvider, ByVal Channel As Boolean) As String
+            Return String.Format(UrlPatternUser, GetUserUrlPart(User))
+        End Function
         Private Const UserRegexDefault As String = "/(profiles|[\w]*?[-]{0,1}channels)/([^/]+)(\Z|.*?)"
         Private Const URD As String = ".*?{0}{1}"
         Friend Overrides Function IsMyUser(ByVal UserURL As String) As ExchangeOptions
@@ -141,7 +145,7 @@ Namespace API.XVIDEOS
                 Dim f As SFile = GetSpecialDataFile(Path, AskForPath, spf)
                 f.Name = "video"
                 f.Extension = "mp4"
-                Using resp As Response = Responser.Copy
+                Using resp As Responser = Responser.Copy
                     Using user As New UserData With {.HOST = Settings(XvideosSiteKey)}
                         DirectCast(user, UserDataBase).User.File = f
                         Dim p As UserMedia = user.Download(URL, resp, DownloadUHD.Value, String.Empty)

SCrawler/API/XVIDEOS/UserData.vb

@@ -45,14 +45,7 @@ Namespace API.XVIDEOS
             UseInternalM3U8Function = True
         End Sub
         Protected Overrides Sub DownloadDataF(ByVal Token As CancellationToken)
-            If Not Settings.UseM3U8 Then
+            If Not Settings.UseM3U8 Then MyMainLOG = $"{ToStringForLog()}: File [ffmpeg.exe] not found" : Exit Sub
-                If Not Settings.OS64 Then
-                    MyMainLOG = $"XVIDEOS [{ToStringForLog()}]: The plugin only works with x64 OS."
-                Else
-                    MyMainLOG = $"{ToStringForLog()}: File [ffmpeg.exe] not found"
-                End If
-                Exit Sub
-            End If
             If IsSavedPosts Then
                 If Not ACheck(MySettings.SavedVideosPlaylist.Value) Then Throw New ArgumentNullException("SavedVideosPlaylist", "Playlist of saved videos cannot be null")
                 DownloadSavedVideos(Token)
@@ -63,49 +56,63 @@ Namespace API.XVIDEOS
         Private Sub DownloadUserVideo(ByVal Token As CancellationToken)
             Dim URL$ = String.Empty
             Try
-                Dim NextPage% = 0
+                Dim NextPage%, d%
-                Dim r$
+                Dim limit% = If(DownloadTopCount, -1)
+                Dim r$, n$
                 Dim j As EContainer = Nothing
                 Dim jj As EContainer
-                Dim user$ = MySettings.GetUserUrl(Me, False)
+                Dim user$ = MySettings.GetUserUrlPart(Me)
                 Dim p As UserMedia
                 Dim EnvirSet As Boolean = False
 
-                Do
+                If ID.IsEmptyString Then GetUserID()
-                    ThrowAny(Token)
+                For i% = 0 To 1
-                    URL = $"https://www.xvideos.com/{user}/videos/new/{If(NextPage = 0, String.Empty, NextPage)}"
+                    If i = 1 And ID.IsEmptyString Then Exit For
-                    r = Responser.GetResponse(URL)
+                    NextPage = 0
-                    If Not r.IsEmptyString Then
+                    d = 0
-                        If Not EnvirSet Then UserExists = True : UserSuspended = False : EnvirSet = True
+                    n = IIf(i = 0, "u", "url")
-                        j = JsonDocument.Parse(r).XmlIfNothing
+                    Do
-                        With j
+                        ThrowAny(Token)
-                            If .Contains("videos") Then
+                        If i = 0 Then
-                                With .Item("videos")
+                            URL = $"https://www.xvideos.com/{user}/videos/new/{If(NextPage = 0, String.Empty, NextPage)}"
-                                    If .Count > 0 Then
+                        Else 'Quickies
-                                        NextPage += 1
+                            URL = $"https://www.xvideos.com/quickies-api/profilevideos/all/none/N/{ID}/{NextPage}"
-                                        For Each jj In .Self
+                        End If
-                                            p = New UserMedia With {
+                        r = Responser.GetResponse(URL,, EDP.ReturnValue)
-                                                .Post = jj.Value("id"),
+                        If Not r.IsEmptyString Then
-                                                .URL = $"https://www.xvideos.com/{jj.Value("u").StringTrimStart("/")}"
+                            If Not EnvirSet Then UserExists = True : UserSuspended = False : EnvirSet = True
-                                            }
+                            j = JsonDocument.Parse(r).XmlIfNothing
-                                            If Not p.Post.ID.IsEmptyString And Not jj.Value("u").IsEmptyString Then
+                            With j
-                                                If Not _TempPostsList.Contains(p.Post.ID) Then
+                                If .Contains("videos") Then
-                                                    _TempPostsList.Add(p.Post.ID)
+                                    With .Item("videos")
-                                                    _TempMediaList.Add(p)
+                                        If .Count > 0 Then
-                                                Else
+                                            NextPage += 1
-                                                    Exit Do
+                                            For Each jj In .Self
+                                                p = New UserMedia With {
+                                                    .Post = jj.Value("id"),
+                                                    .URL = $"https://www.xvideos.com/{jj.Value(n).StringTrimStart("/")}"
+                                                }
+                                                If Not p.Post.ID.IsEmptyString And Not jj.Value(n).IsEmptyString Then
+                                                    If Not _TempPostsList.Contains(p.Post.ID) Then
+                                                        _TempPostsList.Add(p.Post.ID)
+                                                        _TempMediaList.Add(p)
+                                                        d += 1
+                                                        If limit > 0 And d = limit Then Exit Do
+                                                    Else
+                                                        Exit Do
+                                                    End If
                                                 End If
-                                            End If
+                                            Next
-                                        Next
+                                            Continue Do
-                                        Continue Do
+                                        End If
-                                    End If
+                                    End With
-                                End With
+                                End If
-                            End If
+                            End With
-                        End With
+                        End If
-                    End If
+                        If Not j Is Nothing Then j.Dispose()
-                    If Not j Is Nothing Then j.Dispose()
+                        Exit Do
-                    Exit Do
+                    Loop While NextPage < 100
-                Loop While NextPage < 100
+                Next
 
                 If Not j Is Nothing Then j.Dispose()
 
@@ -128,6 +135,10 @@ Namespace API.XVIDEOS
                 If _TempMediaList.ListExists Then _TempMediaList.RemoveAll(Function(m) m.URL.IsEmptyString)
             End Try
         End Sub
+        Private Sub GetUserID()
+            Dim r$ = Responser.GetResponse($"https://www.xvideos.com/{MySettings.GetUserUrlPart(Me)}",, EDP.ReturnValue)
+            If Not r.IsEmptyString Then ID = RegexReplace(r, RParams.DMS("""id_user"":(\d+)", 1, EDP.ReturnValue))
+        End Sub
         Private Sub DownloadSavedVideos(ByVal Token As CancellationToken)
             Dim URL$ = MySettings.SavedVideosPlaylist.Value
             Try
@@ -171,7 +182,7 @@ Namespace API.XVIDEOS
                 ProcessException(ex, Token, $"data downloading error [{URL}]")
             End Try
         End Sub
-        Private Function GetVideoData(ByVal Media As UserMedia, ByVal resp As Response, ByVal DownloadUHD As Boolean) As UserMedia
+        Private Function GetVideoData(ByVal Media As UserMedia, ByVal resp As Responser, ByVal DownloadUHD As Boolean) As UserMedia
             Try
                 If Not Media.URL.IsEmptyString Then
                     Dim r$ = resp.GetResponse(Media.URL)
@@ -217,7 +228,7 @@ Namespace API.XVIDEOS
                 Return Nothing
             End Try
         End Function
-        Friend Function Download(ByVal URL As String, ByVal resp As Response, ByVal DownloadUHD As Boolean, ByVal ID As String)
+        Friend Function Download(ByVal URL As String, ByVal resp As Responser, ByVal DownloadUHD As Boolean, ByVal ID As String)
             Dim m As UserMedia = GetVideoData(New UserMedia(URL, UTypes.VideoPre) With {.Post = ID}, resp, DownloadUHD)
             If Not m.URL.IsEmptyString Then
                 Dim f As SFile = m.File

SCrawler/API/Xhamster/M3U8.vb

@@ -14,7 +14,7 @@ Namespace API.Xhamster
     Friend NotInheritable Class M3U8
         Private Sub New()
         End Sub
-        Private Shared Function ParseFirstM3U8(ByVal URL As String, ByVal Responser As Response, ByVal UHD As Boolean) As String
+        Private Shared Function ParseFirstM3U8(ByVal URL As String, ByVal Responser As Responser, ByVal UHD As Boolean) As String
             Dim r$, d$
             Dim _DataObtained As Boolean = False
             For i% = 0 To 1
@@ -38,7 +38,7 @@ Namespace API.Xhamster
             Next
             Return String.Empty
         End Function
-        Private Shared Function ParseSecondM3U8(ByVal URL As String, ByVal Responser As Response, ByVal Appender As String) As List(Of String)
+        Private Shared Function ParseSecondM3U8(ByVal URL As String, ByVal Responser As Responser, ByVal Appender As String) As List(Of String)
             Dim r$
             Dim l As List(Of String)
             For i% = 0 To 1
@@ -57,7 +57,7 @@ Namespace API.Xhamster
             Next
             Return Nothing
         End Function
-        Private Shared Function ObtainUrls(ByVal URL As String, ByVal Responser As Response, ByVal UHD As Boolean) As List(Of String)
+        Private Shared Function ObtainUrls(ByVal URL As String, ByVal Responser As Responser, ByVal UHD As Boolean) As List(Of String)
             Try
                 Dim file$ = ParseFirstM3U8(URL, Responser, UHD)
                 If Not file.IsEmptyString Then
@@ -72,7 +72,7 @@ Namespace API.Xhamster
                 Responser.UseGZipStream = False
             End Try
         End Function
-        Friend Shared Function Download(ByVal Media As UserMedia, ByVal Responser As Response, ByVal UHD As Boolean) As SFile
+        Friend Shared Function Download(ByVal Media As UserMedia, ByVal Responser As Responser, ByVal UHD As Boolean) As SFile
             Return M3U8Base.Download(ObtainUrls(Media.URL, Responser, UHD), Media.File, Responser)
         End Function
     End Class

SCrawler/API/Xhamster/SiteSettings.vb

@@ -102,7 +102,7 @@ Namespace API.Xhamster
         End Function
         Friend Overrides Function GetSpecialData(ByVal URL As String, ByVal Path As String, ByVal AskForPath As Boolean) As IEnumerable
             If Available(ISiteSettings.Download.Main, True) Then
-                Using resp As Response = Responser.Copy
+                Using resp As Responser = Responser.Copy
                     Dim spf$ = String.Empty
                     Dim f As SFile = GetSpecialDataFile(Path, AskForPath, spf)
                     Dim m As UserMedia = UserData.GetVideoInfo(URL, resp, f)
@@ -114,10 +114,10 @@ Namespace API.Xhamster
             End If
             Return Nothing
         End Function
-        Friend Overrides Function Available(ByVal What As ISiteSettings.Download, Silent As Boolean) As Boolean
+        Friend Overrides Function Available(ByVal What As ISiteSettings.Download, ByVal Silent As Boolean) As Boolean
             If Settings.UseM3U8 AndAlso MyBase.Available(What, Silent) Then
                 If What = ISiteSettings.Download.SavedPosts Then
-                    Return If(Responser.Cookies?.Count, 0) > 0
+                    Return Responser.CookiesExists
                 Else
                     Return True
                 End If

SCrawler/API/Xhamster/UserData.vb

@@ -219,7 +219,7 @@ Namespace API.Xhamster
         End Sub
 #End Region
 #Region "GetM3U8"
-        Private Overloads Function GetM3U8(ByRef m As UserMedia, ByVal URL As String, ByVal Responser As Response,
+        Private Overloads Function GetM3U8(ByRef m As UserMedia, ByVal URL As String, ByVal Responser As Responser,
                                            Optional ByVal e As ErrorsDescriber = Nothing) As Boolean
             Try
                 If Not URL.IsEmptyString Then
@@ -248,7 +248,7 @@ Namespace API.Xhamster
         End Function
 #End Region
 #Region "Standalone downloader"
-        Friend Shared Function GetVideoInfo(ByVal URL As String, ByVal Responser As Response, ByVal Path As SFile) As UserMedia
+        Friend Shared Function GetVideoInfo(ByVal URL As String, ByVal Responser As Responser, ByVal Path As SFile) As UserMedia
             Try
                 Using u As New UserData With {.Responser = Responser, .HOST = Settings(XhamsterSiteKey)}
                     Dim m As UserMedia = Nothing

SCrawler/Download/Automation/AutoDownloader.vb

@@ -143,7 +143,8 @@ Namespace DownloadObjects
             ''' <returns>True to activate</returns>
             Friend Function Open(ByVal _Key As String) As Boolean
                 If Not User Is Nothing Then
-                    If Key = _Key Then
+                    If KeyDismiss = _Key Then
+                    ElseIf Key = _Key Then
                         Return True
                     ElseIf KeyFolder = _Key Then
                         User.OpenFolder()
@@ -152,6 +153,8 @@ Namespace DownloadObjects
                     ElseIf Images.ContainsKey(_Key) Then
                         Images(_Key).Open(, EDP.None)
                     End If
+                Else
+                    Return True
                 End If
                 Return False
             End Function
@@ -471,18 +474,25 @@ Namespace DownloadObjects
                 Dim DownloadedUsersCount% = 0
                 Dim simple As Boolean = ShowSimpleNotification And ShowNotifications
                 Dim notify As Action = Sub()
-                                           With Downloader.Downloaded
+                                           Try
-                                               If ShowNotifications And .Count > 0 Then .ForEach(Sub(ByVal u As IUserData)
+                                               With Downloader.Downloaded
-                                                                                                     If Keys.Contains(u.Key) Then
+                                                   If ShowNotifications And .Count > 0 Then
-                                                                                                         If simple Then
+                                                       For indx% = 0 To .Count - 1
-                                                                                                             DownloadedUsersCount += 1
+                                                           With .Item(indx)
-                                                                                                         Else
+                                                               If Keys.Contains(.Key) Then
-                                                                                                             ShowNotification(u)
+                                                                   If simple Then
-                                                                                                         End If
+                                                                       DownloadedUsersCount += 1
-                                                                                                         Keys.Remove(u.Key)
+                                                                   Else
-                                                                                                     End If
+                                                                       ShowNotification(.Self)
-                                                                                                 End Sub)
+                                                                   End If
-                                           End With
+                                                                   Keys.Remove(.Key)
+                                                               End If
+                                                           End With
+                                                       Next
+                                                   End If
+                                               End With
+                                           Catch n_ex As Exception
+                                           End Try
                                        End Sub
                 Select Case Mode
                     Case Modes.All
@@ -541,10 +551,12 @@ Namespace DownloadObjects
                 UserKeys.Last.ShowNotification()
             End If
         End Sub
-        Friend Function NotificationClicked(ByVal Key As String) As Boolean
+        Friend Function NotificationClicked(ByVal Key As String, ByRef Found As Boolean, ByRef ActivateForm As Boolean) As Boolean
             Dim i% = UserKeys.IndexOf(Key)
             If i >= 0 Then
-                MainFrameObj.FocusUser(UserKeys(i).IUserDataKey, UserKeys(i).Open(Key))
+                Found = True
+                ActivateForm = UserKeys(i).Open(Key)
+                MainFrameObj.FocusUser(UserKeys(i).IUserDataKey, ActivateForm)
                 Return True
             Else
                 Return False

SCrawler/Download/Automation/Scheduler.vb

@@ -52,8 +52,13 @@ Namespace DownloadObjects
                 Return Plans.Count
             End Get
         End Property
-        Friend Function NotificationClicked(ByVal Key As String) As Boolean
+        Friend Function NotificationClicked(ByVal Key As String, ByRef Found As Boolean, ByRef ActivateForm As Boolean) As Boolean
-            Return Count > 0 AndAlso Plans.Exists(Function(p) p.NotificationClicked(Key))
+            If Count > 0 Then
+                For Each plan As AutoDownloader In Plans
+                    If plan.NotificationClicked(Key, Found, ActivateForm) Then Return True
+                Next
+            End If
+            Return False
         End Function
         Friend Sub Add(ByVal Plan As AutoDownloader)
             Plan.Source = Me

SCrawler/Download/DownloadedInfoForm.vb

@@ -150,8 +150,8 @@ Namespace DownloadObjects
         Private Sub BTT_FIND_Click(sender As Object, e As EventArgs) Handles BTT_FIND.Click
             Try
                 If _LatestSelected.ValueBetween(0, LIST_DOWN.Items.Count - 1) AndAlso _LatestSelected.ValueBetween(0, Downloader.Downloaded.Count - 1) Then
-                    Dim i% = Settings.Users.IndexOf(_TempUsersList(_LatestSelected))
+                    Dim u As IUserData = Settings.GetUser(_TempUsersList(_LatestSelected), True)
-                    If i >= 0 Then RaiseEvent UserFind(Settings.Users(i).Key)
+                    If Not u Is Nothing Then RaiseEvent UserFind(u.Key)
                 End If
             Catch ex As Exception
             End Try

SCrawler/Download/Feed/FeedMedia.vb

@@ -115,6 +115,7 @@ Namespace DownloadObjects
                         Case UserMedia.Types.Video, UserMedia.Types.m3u8
                             infoType = UserMedia.Types.Video
                             MyVideo = New FeedVideo(File) With {.Tag = File, .Dock = DockStyle.Fill, .ContextMenuStrip = CONTEXT_DATA}
+                            If MyVideo.HasError Then HasError = True
                             TP_MAIN.Controls.Add(MyVideo, 0, 1)
                             BTT_CONTEXT_OPEN_MEDIA.Text &= " video"
                             BTT_CONTEXT_DELETE.Text &= " video"

SCrawler/Download/Feed/FeedVideo.vb

@@ -38,6 +38,7 @@ Namespace DownloadObjects
         Private VideoLengthMs As Integer = 0
         Private VideoLengthStr As String
         Private MediaFile As SFile = Nothing
+        Friend ReadOnly HasError As Boolean = False
         Public Sub New()
             InitializeComponent()
         End Sub
@@ -63,7 +64,6 @@ Namespace DownloadObjects
                                 MyVideo.BackgroundImageLayout = ImageLayout.Zoom
                             End If
                         Catch img_set_ex As Exception
-                            'TODELETE: FeedVideo set BackgroundImage error
                             ErrorsDescriber.Execute(EDP.SendInLog, img_set_ex, "Error setting background image for media player." & vbCr &
                                                                                $"File: {File}{vbCr}Image: {f}")
                         End Try
@@ -71,8 +71,8 @@ Namespace DownloadObjects
                 End If
                 UpdateButtons()
             Catch ex As Exception
-                'TODELETE: FeedVideo initialization error
                 ErrorsDescriber.Execute(EDP.SendInLog, ex, $"Media player initialization error({File})")
+                HasError = True
             End Try
         End Sub
         Private _Disposed As Boolean = False

SCrawler/Download/Groups/DownloadGroupCollection.vb

@@ -34,9 +34,6 @@ Namespace DownloadObjects.Groups
             End If
             GroupsList.ListReindex
         End Sub
-        Friend Function GetLabels() As List(Of String)
-            Return ListAddList(Nothing, GroupsList.SelectMany(Function(g) g.Labels), LAP.NotContainsOnly)
-        End Function
         Default Friend ReadOnly Property Item(ByVal Index As Integer) As DownloadGroup Implements IMyEnumerator(Of DownloadGroup).MyEnumeratorObject
             Get
                 Return GroupsList(Index)

SCrawler/Download/VideosDownloaderForm.Designer.vb

@@ -151,10 +151,10 @@ Namespace DownloadObjects
             Me.Controls.Add(Me.LIST_VIDEOS)
             Me.Controls.Add(Me.ToolbarBOTTOM)
             Me.Controls.Add(Me.ToolbarTOP)
+            Me.Icon = Global.SCrawler.My.Resources.Resources.ArrowDownIcon_Blue_24
             Me.KeyPreview = True
             Me.MinimumSize = New System.Drawing.Size(540, 400)
             Me.Name = "VideosDownloaderForm"
-            Me.ShowIcon = False
             Me.Text = "Download videos"
             Me.ToolbarTOP.ResumeLayout(False)
             Me.ToolbarTOP.PerformLayout()

SCrawler/Download/WebClient2.vb

@@ -11,12 +11,12 @@ Imports PersonalUtilities.Tools.Web.Clients
 Namespace DownloadObjects
     Friend Class WebClient2 : Implements IDisposable
         Protected WC As WebClient
-        Protected RC As Response
+        Protected RC As Responser
         Private ReadOnly RCERROR As New ErrorsDescriber(EDP.ThrowException)
         Protected UseResponserClient As Boolean
         Friend Sub New()
         End Sub
-        Friend Sub New(ByVal Responser As Response)
+        Friend Sub New(ByVal Responser As Responser)
             If Not Responser Is Nothing Then
                 RC = Responser
                 UseResponserClient = True

SCrawler/Editors/LabelsForm.vb

@@ -72,7 +72,7 @@ Friend Class LabelsForm
     Private Sub MyDefs_ButtonOkClick(ByVal Sender As Object, ByVal e As KeyHandleEventArgs) Handles MyDefs.ButtonOkClick
         Try
             LabelsList.ListAddList(CMB_LABELS.Items.CheckedItems.Select(Function(l) CStr(l.Value(0))), LAP.ClearBeforeAdd, LAP.NotContainsOnly)
-            If _AnyLabelAdd And _Source Is Nothing Then Settings.Labels.Update()
+            If _Source Is Nothing Then Settings.Labels.Update()
             MyDefs.CloseForm()
         Catch ex As Exception
             ErrorsDescriber.Execute(EDP.LogMessageValue, ex, "Label selection")

SCrawler/Editors/SiteEditorForm.vb

@@ -12,7 +12,6 @@ Imports PersonalUtilities.Forms
 Imports PersonalUtilities.Forms.Controls
 Imports PersonalUtilities.Forms.Controls.Base
 Imports PersonalUtilities.Tools.Web.Cookies
-Imports CookieControl = PersonalUtilities.Tools.Web.Cookies.CookieListForm.CookieControl
 Imports ADB = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons
 Namespace Editors
     Friend Class SiteEditorForm
@@ -213,12 +212,13 @@ Namespace Editors
             Select Case Sender.DefaultButton
                 Case ADB.Edit
                     If Not Host.Responser Is Nothing Then
-                        Using f As New CookieListForm(Host.Responser) With {
+                        Using f As New CookieListForm With {.DesignXML = Settings.Design, .UseGrid = False}
-                            .MyDesignXML = Settings.Design,
+                            f.SetCollection(Host.Responser.Cookies)
-                            .DisableControls = CookieControl.AddFromInternal + CookieControl.AuthorizeProgram + CookieControl.OpenBrowser
-                        }
                             f.ShowDialog()
-                            MyDefs.MyOkCancel.EnableOK = True
+                            If f.DialogResult = DialogResult.OK Then
+                                f.GetCollection(Host.Responser)
+                                MyDefs.MyOkCancel.EnableOK = True
+                            End If
                         End Using
                         SetCookieText()
                     End If

SCrawler/Editors/UserCreatorForm.Designer.vb

@@ -106,10 +106,10 @@ Namespace Editors
             'BTT_OTHER_SETTINGS
             '
             Me.BTT_OTHER_SETTINGS.Dock = System.Windows.Forms.DockStyle.Fill
-            Me.BTT_OTHER_SETTINGS.Location = New System.Drawing.Point(330, 2)
+            Me.BTT_OTHER_SETTINGS.Location = New System.Drawing.Point(329, 2)
             Me.BTT_OTHER_SETTINGS.Margin = New System.Windows.Forms.Padding(1)
             Me.BTT_OTHER_SETTINGS.Name = "BTT_OTHER_SETTINGS"
-            Me.BTT_OTHER_SETTINGS.Size = New System.Drawing.Size(120, 24)
+            Me.BTT_OTHER_SETTINGS.Size = New System.Drawing.Size(121, 24)
             Me.BTT_OTHER_SETTINGS.TabIndex = 2
             Me.BTT_OTHER_SETTINGS.Text = "Options (F2)"
             TT_MAIN.SetToolTip(Me.BTT_OTHER_SETTINGS, "Other settings")
@@ -121,7 +121,7 @@ Namespace Editors
             'CONTAINER_MAIN.ContentPanel
             '
             CONTAINER_MAIN.ContentPanel.Controls.Add(Me.TP_MAIN)
-            CONTAINER_MAIN.ContentPanel.Size = New System.Drawing.Size(454, 436)
+            CONTAINER_MAIN.ContentPanel.Size = New System.Drawing.Size(454, 461)
             CONTAINER_MAIN.Dock = System.Windows.Forms.DockStyle.Fill
             CONTAINER_MAIN.LeftToolStripPanelVisible = False
             CONTAINER_MAIN.Location = New System.Drawing.Point(0, 0)
@@ -162,7 +162,7 @@ Namespace Editors
             Me.TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 26.0!))
             Me.TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 26.0!))
             Me.TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Percent, 100.0!))
-            Me.TP_MAIN.Size = New System.Drawing.Size(454, 436)
+            Me.TP_MAIN.Size = New System.Drawing.Size(454, 461)
             Me.TP_MAIN.TabIndex = 0
             '
             'TXT_USER
@@ -182,7 +182,7 @@ Namespace Editors
             Me.TP_SITE.ColumnCount = 3
             Me.TP_SITE.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Absolute, 79.0!))
             Me.TP_SITE.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Percent, 100.0!))
-            Me.TP_SITE.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Absolute, 122.0!))
+            Me.TP_SITE.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Absolute, 123.0!))
             Me.TP_SITE.Controls.Add(Me.CH_IS_CHANNEL, 0, 0)
             Me.TP_SITE.Controls.Add(Me.CMB_SITE, 1, 0)
             Me.TP_SITE.Controls.Add(Me.BTT_OTHER_SETTINGS, 2, 0)
@@ -225,7 +225,7 @@ Namespace Editors
             Me.CMB_SITE.Location = New System.Drawing.Point(84, 3)
             Me.CMB_SITE.Margin = New System.Windows.Forms.Padding(3, 2, 3, 3)
             Me.CMB_SITE.Name = "CMB_SITE"
-            Me.CMB_SITE.Size = New System.Drawing.Size(241, 22)
+            Me.CMB_SITE.Size = New System.Drawing.Size(240, 22)
             Me.CMB_SITE.TabIndex = 1
             Me.CMB_SITE.TextBoxBorderStyle = System.Windows.Forms.BorderStyle.FixedSingle
             '
@@ -302,7 +302,7 @@ Namespace Editors
             Me.TXT_DESCR.Location = New System.Drawing.Point(4, 290)
             Me.TXT_DESCR.Multiline = True
             Me.TXT_DESCR.Name = "TXT_DESCR"
-            Me.TXT_DESCR.Size = New System.Drawing.Size(446, 142)
+            Me.TXT_DESCR.Size = New System.Drawing.Size(446, 167)
             Me.TXT_DESCR.TabIndex = 10
             '
             'TXT_USER_FRIENDLY
@@ -469,7 +469,7 @@ Namespace Editors
             Me.Name = "UserCreatorForm"
             Me.ShowInTaskbar = False
             Me.SizeGripStyle = System.Windows.Forms.SizeGripStyle.Hide
-            Me.Text = "Create User"
+            Me.Text = "Create user"
             CONTAINER_MAIN.ContentPanel.ResumeLayout(False)
             CONTAINER_MAIN.ResumeLayout(False)
             CONTAINER_MAIN.PerformLayout()

SCrawler/Editors/UserCreatorForm.vb

@@ -241,6 +241,8 @@ Namespace Editors
                             CMB_SITE.Enabled = False
                             CH_IS_CHANNEL.Checked = User.IsChannel
                             If Not UserInstance Is Nothing Then
+                                Text = $"User: {UserInstance.Name}"
+                                If Not UserInstance.FriendlyName.IsEmptyString Then Text &= $" ({UserInstance.FriendlyName})"
                                 TXT_USER.Enabled = False
                                 TXT_SPEC_FOLDER.TextBoxReadOnly = True
                                 TXT_SPEC_FOLDER.Buttons.Clear()
@@ -395,6 +397,7 @@ CloseForm:
                             CMB_SITE.SelectedIndex = -1
                             CMB_SITE.Clear(ComboBoxExtended.ClearMode.Text)
                             CH_IS_CHANNEL.Checked = False
+                            If Not UserIsCollection Then Icon = My.Resources.UsersIcon_32
                         End If
                     End If
                     _TextChangeInvoked = False
@@ -410,6 +413,9 @@ CloseForm:
             MyExchangeOptions = Nothing
             SetParamsBySite()
         End Sub
+        Private Sub CMB_SITE_ActionOnTextChanged(sender As Object, e As EventArgs) Handles CMB_SITE.ActionOnTextChanged
+            If CMB_SITE.Text.IsEmptyString And Not UserIsCollection Then CMB_SITE.SelectedIndex = -1 : Icon = My.Resources.UsersIcon_32
+        End Sub
         Private Sub BTT_OTHER_SETTINGS_Click(sender As Object, e As EventArgs) Handles BTT_OTHER_SETTINGS.Click
             Dim s As SettingsHost = GetSiteByCheckers()
             If Not s Is Nothing Then
@@ -602,9 +608,17 @@ CloseForm:
                     Else
                         BTT_OTHER_SETTINGS.Enabled = False
                     End If
+                    If Not UserIsCollection Then
+                        If Not s.Source.Icon Is Nothing Then
+                            Icon = s.Source.Icon
+                        ElseIf Not s.Source.Image Is Nothing Then
+                            Icon = ImageRenderer.GetIcon(s.Source.Image, New ErrorsDescriber(False, False, False, My.Resources.UsersIcon_32))
+                        End If
+                    End If
                 End With
             Else
                 BTT_OTHER_SETTINGS.Enabled = False
+                If Not UserIsCollection Then Icon = My.Resources.UsersIcon_32
             End If
         End Sub
         Private Sub ChangeLabels()

SCrawler/EncryptCookies.vb

@@ -10,7 +10,7 @@ Imports PersonalUtilities.Tools.Web.Clients
 Namespace EncryptCookies
     Friend Module EncryptFunction
         Friend CookiesEncrypted As Boolean = False
-        Friend Sub ValidateCookiesEncrypt(ByRef Responser As Response)
+        Friend Sub ValidateCookiesEncrypt(ByRef Responser As Responser)
             If Not Responser Is Nothing Then
                 Dim b As Boolean = False
                 With Responser

SCrawler/LabelsKeeper.vb

@@ -25,30 +25,16 @@ Friend Class LabelsKeeper : Implements ICollection(Of String), IMyEnumerator(Of
     Friend ReadOnly Property Current As XMLValuesCollection(Of String)
     Friend ReadOnly Property Excluded As XMLValuesCollection(Of String)
     Friend ReadOnly Property ExcludedIgnore As XMLValue(Of Boolean)
-    Private ReadOnly Property SourceXML As XmlFile
     Friend Sub New(ByRef x As XmlFile)
-        SourceXML = x
         LabelsList = New List(Of String)
         NewLabels = New List(Of String)
         If LabelsFile.Exists Then LabelsList.ListAddList(IO.File.ReadAllLines(LabelsFile), LAP.NotContainsOnly)
         Current = New XMLValuesCollection(Of String)(XMLValueBase.ListModes.String, "LatestSelectedLabels", x) With {.ListAddParameters = LAP.NotContainsOnly}
         Excluded = New XMLValuesCollection(Of String)(XMLValueBase.ListModes.String, "LatestExcludedLabels", x) With {.ListAddParameters = LAP.NotContainsOnly}
         ExcludedIgnore = New XMLValue(Of Boolean)("LatestExcludedLabelsIgnore", False, x)
-    End Sub
+        Dim lp As New ListAddParams(LAP.NotContainsOnly + LAP.IgnoreICopier)
-    Friend Sub Verify()
+        If Current.Count > 0 Then LabelsList.ListAddList(Current, lp)
-        SourceXML.BeginUpdate()
+        If Excluded.Count > 0 Then LabelsList.ListAddList(Excluded, lp)
-        Dim r As Predicate(Of String) = Function(l) Not LabelsList.Contains(l)
-        Dim c% = Current.Count
-        If c > 0 Then
-            Current.ValuesList.RemoveAll(r)
-            If Not Current.Count = c Then Current.Update()
-        End If
-        c = Excluded.Count
-        If c > 0 Then
-            Excluded.ValuesList.RemoveAll(r)
-            If Not c = Excluded.Count Then Excluded.Update()
-        End If
-        SourceXML.EndUpdate()
     End Sub
     Friend ReadOnly Property ToList As List(Of String)
         Get
@@ -69,10 +55,14 @@ Friend Class LabelsKeeper : Implements ICollection(Of String), IMyEnumerator(Of
         LabelsList.Clear()
         NewLabels.Clear()
     End Sub
-    Friend Sub Update()
+    Friend Sub Update(Optional ByVal Force As Boolean = False)
         If LabelsList.Count > 0 Then
-            LabelsList.Sort()
+            If NewLabelsExists Or Force Then
-            TextSaver.SaveTextToFile(LabelsList.ListToString(vbNewLine), LabelsFile, True, False, EDP.SendInLog)
+                If LabelsList.Contains(NoParsedUser) Then LabelsList.Remove(NoParsedUser)
+                LabelsList.Sort()
+                TextSaver.SaveTextToFile(LabelsList.ListToString(vbNewLine), LabelsFile, True, False, EDP.SendInLog)
+                If NewLabels.Count > 0 Then NewLabels.Clear()
+            End If
         Else
             LabelsFile.Delete(, Settings.DeleteMode, EDP.SendInLog)
         End If

SCrawler/ListImagesLoader.vb

@@ -6,11 +6,12 @@
 '
 ' This program is distributed in the hope that it will be useful,
 ' but WITHOUT ANY WARRANTY
+Imports System.Threading
 Imports SCrawler.API
 Imports SCrawler.API.Base
 Friend Class ListImagesLoader
     Private ReadOnly Property MyList As ListView
-    Private Class UserOption : Implements IComparable(Of UserOption)
+    Private Structure UserOption : Implements IComparable(Of UserOption)
         Friend ReadOnly User As IUserData
         Friend ReadOnly LVI As ListViewItem
         Friend Index As Integer
@@ -19,95 +20,133 @@ Friend Class ListImagesLoader
                 Return LVI.Name
             End Get
         End Property
-        Friend [Image] As Image
+        Friend Sub New(ByVal u As IUserData, ByVal l As ListView)
-        Friend Sub New(ByVal u As IUserData, ByVal l As ListView, ByVal GetImage As Boolean)
             User = u
             LVI = u.GetLVI(l)
             Index = u.Index
-            If GetImage Then Image = u.GetPicture
-        End Sub
-        Friend Sub UpdateImage()
-            Image = User.GetPicture
         End Sub
         Friend Function CompareTo(ByVal Other As UserOption) As Integer Implements IComparable(Of UserOption).CompareTo
             Return Index.CompareTo(Other.Index)
         End Function
-    End Class
+    End Structure
     Friend Sub New(ByRef l As ListView)
         MyList = l
     End Sub
-    Friend Sub Update()
+    Private UserDataList As List(Of UserOption)
-        Dim a As Action = Sub()
+    Private UpdateInProgress As Boolean = False
-                              With MyList
+    Private ImageThread As Thread
-                                  .Items.Clear()
+    Private Sub UpdateImages()
-                                  If Not .LargeImageList Is Nothing Then .LargeImageList.Images.Clear()
+        If UserDataList.ListExists And Not If(ImageThread?.IsAlive, False) Then
-                                  .LargeImageList = New ImageList
+            ImageThread = New Thread(New ThreadStart(Sub()
-                                  If Not .SmallImageList Is Nothing Then .SmallImageList.Images.Clear()
+                                                         Dim ar As IAsyncResult = Nothing
-                                  .SmallImageList = New ImageList
+                                                         Dim a As Action = Sub()
-                                  If Settings.ViewModeIsPicture Then
+                                                                               Try
-                                      .LargeImageList.ColorDepth = ColorDepth.Depth32Bit
+                                                                                   If UserDataList.ListExists Then
-                                      .SmallImageList.ColorDepth = ColorDepth.Depth32Bit
+                                                                                       For i% = 0 To UserDataList.Count - 1
-                                      .LargeImageList.ImageSize = New Size(DivideWithZeroChecking(Settings.MaxLargeImageHeight.Value, 100) * 75, Settings.MaxLargeImageHeight.Value)
+                                                                                           With UserDataList(i).User
-                                      .SmallImageList.ImageSize = New Size(DivideWithZeroChecking(Settings.MaxSmallImageHeight.Value, 100) * 75, Settings.MaxSmallImageHeight.Value)
+                                                                                               Select Case Settings.ViewMode.Value
-                                  End If
+                                                                                                   Case View.LargeIcon : MyList.LargeImageList.Images.Add(.Key, .GetPicture())
-                              End With
+                                                                                                   Case View.SmallIcon : MyList.SmallImageList.Images.Add(.Key, .GetPicture())
-                          End Sub
+                                                                                               End Select
-        If MyList.InvokeRequired Then MyList.Invoke(a) Else a.Invoke
+                                                                                           End With
-        If Settings.Users.Count > 0 Then
+                                                                                           Application.DoEvents()
-            Settings.Users.Sort()
+                                                                                       Next
-            Dim v As View = Settings.ViewMode.Value
+                                                                                       UserDataList.Clear()
-            Dim i%
+                                                                                       GC.Collect()
-
+                                                                                   End If
-            With MyList
+                                                                               Catch iex As ArgumentOutOfRangeException
-                MyList.BeginUpdate()
+                                                                               Catch ex As Exception
-
+                                                                                   ErrorsDescriber.Execute(EDP.SendInLog, ex, "[ListImagesLoader.UpdateImages]")
-                If Settings.FastProfilesLoading Then
+                                                                               End Try
-                    Settings.Users.ListReindex
+                                                                               If Not ar Is Nothing Then MyList.EndInvoke(ar)
-                    Dim UData As List(Of UserOption)
+                                                                               UpdateInProgress = False
-
+                                                                           End Sub
-                    If Settings.ViewModeIsPicture Then
+                                                         If MyList.InvokeRequired Then
-                        UData = GetUsersWithImages()
+                                                             ar = MyList.BeginInvoke(a)
-                        If UData.ListExists Then
+                                                         Else
-                            UData.Sort()
+                                                             a.Invoke
-                            Select Case v
+                                                         End If
-                                Case View.LargeIcon : .LargeImageList.Images.AddRange(UData.Select(Function(u) u.Image).ToArray)
+                                                     End Sub)) With {.IsBackground = True}
-                                Case View.SmallIcon : .SmallImageList.Images.AddRange(UData.Select(Function(u) u.Image).ToArray)
+            ImageThread.SetApartmentState(ApartmentState.MTA)
-                            End Select
+            ImageThread.Start()
-                        End If
-                    Else
-                        UData = (From u As IUserData In Settings.Users Where u.FitToAddParams Select New UserOption(u, MyList, False)).ListIfNothing
-                        If UData.ListExists Then UData.Sort()
-                    End If
-
-                    If UData.ListExists Then
-                        If Settings.ViewModeIsPicture Then
-                            For i = 0 To UData.Count - 1
-                                Select Case v
-                                    Case View.LargeIcon : .LargeImageList.Images.SetKeyName(i, UData(i).Key)
-                                    Case View.SmallIcon : .SmallImageList.Images.SetKeyName(i, UData(i).Key)
-                                End Select
-                            Next
-                        End If
-                        .Items.AddRange(UData.Select(Function(u) u.LVI).ToArray)
-                        UData.Clear()
-                    End If
-                Else
-                    Dim t As New List(Of Task)
-                    For Each User As IUserData In Settings.Users
-                        If User.FitToAddParams Then
-                            If Settings.ViewModeIsPicture Then
-                                t.Add(Task.Run(Sub() UpdateUser(User, True)))
-                            Else
-                                UpdateUser(User, True)
-                            End If
-                        End If
-                    Next
-                    If t.Count > 0 Then Task.WhenAll(t.ToArray) : t.Clear()
-                End If
-            End With
-            MyList.EndUpdate()
         End If
     End Sub
+    Private Sub InterruptUpdate()
+        Try
+            If UserDataList.ListExists Then UserDataList.Clear() : Application.DoEvents()
+            If If(ImageThread?.IsAlive, False) Then ImageThread.Abort() : Application.DoEvents()
+        Catch ex As Exception
+            ErrorsDescriber.Execute(EDP.SendInLog, ex, "[ListImagesLoader.InterruptUpdate]")
+        End Try
+    End Sub
+    Friend Sub Update()
+        Try
+            If UpdateInProgress Then InterruptUpdate()
+            If Not UpdateInProgress Then
+                UpdateInProgress = True
+                Dim a As Action = Sub()
+                                      With MyList
+                                          .Items.Clear()
+                                          If Not .LargeImageList Is Nothing Then .LargeImageList.Images.Clear()
+                                          .LargeImageList = New ImageList
+                                          If Not .SmallImageList Is Nothing Then .SmallImageList.Images.Clear()
+                                          .SmallImageList = New ImageList
+                                          If Settings.ViewModeIsPicture Then
+                                              .LargeImageList.ColorDepth = ColorDepth.Depth32Bit
+                                              .SmallImageList.ColorDepth = ColorDepth.Depth32Bit
+                                              .LargeImageList.ImageSize = New Size(DivideWithZeroChecking(Settings.MaxLargeImageHeight.Value, 100) * 75, Settings.MaxLargeImageHeight.Value)
+                                              .SmallImageList.ImageSize = New Size(DivideWithZeroChecking(Settings.MaxSmallImageHeight.Value, 100) * 75, Settings.MaxSmallImageHeight.Value)
+                                          End If
+                                      End With
+                                  End Sub
+                If MyList.InvokeRequired Then MyList.Invoke(a) Else a.Invoke
+                If Settings.Users.Count > 0 Then
+                    Settings.Users.Sort()
+                    Dim v As View = Settings.ViewMode.Value
+
+                    With MyList
+                        MyList.BeginUpdate()
+
+                        If Settings.FastProfilesLoading Then
+                            Settings.Users.ListReindex
+
+                            UserDataList = (From u As IUserData In Settings.Users Where u.FitToAddParams Select New UserOption(u, MyList)).ListIfNothing
+                            If UserDataList.ListExists Then UserDataList.Sort()
+
+                            If UserDataList.ListExists Then
+                                .Items.AddRange(UserDataList.Select(Function(u) u.LVI).ToArray)
+                                If Settings.ViewModeIsPicture Then
+                                    MyList.EndUpdate()
+                                    UpdateImages()
+                                Else
+                                    UserDataList.Clear()
+                                    UpdateInProgress = False
+                                End If
+                            End If
+                        Else
+                            Dim t As New List(Of Task)
+                            For Each User As IUserData In Settings.Users
+                                If User.FitToAddParams Then
+                                    If Settings.ViewModeIsPicture Then
+                                        t.Add(Task.Run(Sub() UpdateUser(User, True)))
+                                    Else
+                                        UpdateUser(User, True)
+                                    End If
+                                End If
+                            Next
+                            If t.Count > 0 Then Task.WhenAll(t.ToArray) : t.Clear()
+                            UpdateInProgress = False
+                        End If
+                    End With
+                    MyList.EndUpdate()
+                End If
+            Else
+                MsgBoxE({"User list update aborted. Click the 'Refresh' button to refresh the user list.", "Update user list"}, vbExclamation)
+            End If
+        Catch ex As Exception
+            ErrorsDescriber.Execute(EDP.SendInLog, ex, "[ListImagesLoader.Update]")
+        End Try
+    End Sub
     Friend Sub UpdateUser(ByVal User As IUserData, ByVal Add As Boolean)
         Try
             Dim a As Action
@@ -172,24 +211,4 @@ Friend Class ListImagesLoader
             Return False
         End If
     End Function
-    Private Function GetUsersWithImages() As List(Of UserOption)
-        Try
-            Dim t As New List(Of Task)
-            Dim l As New List(Of UserOption)
-            t.AddRange(From u As IUserData In Settings.Users Where u.FitToAddParams Select Task.Run(Sub() l.Add(New UserOption(u, MyList, True))))
-            If t.Count > 0 Then Task.WaitAll(t.ToArray)
-            If l.Count > 0 Then
-                For i% = 0 To l.Count - 1
-                    If l(i) Is Nothing Then Throw New ArgumentNullException("UserOption", $"One of the UserOptions [{i} / {l.Count - 1}] is null.")
-                    If l(i).Image Is Nothing Then l(i).UpdateImage()
-                Next
-            End If
-            Return l
-        Catch ex As Exception
-            Return ErrorsDescriber.Execute(EDP.LogMessageValue, ex,
-                                           "Image fast loading error." & vbCr &
-                                           "Click the ""Refresh"" button to manually refresh the user list." & vbCr &
-                                           "[ListImagesLoader.GetUsersWithImages]")
-        End Try
-    End Function
 End Class

SCrawler/MainFrame.Designer.vb

@@ -121,6 +121,7 @@ Partial Public Class MainFrame : Inherits System.Windows.Forms.Form
         Me.BTT_TRAY_PAUSE_AUTOMATION = New System.Windows.Forms.ToolStripMenuItem()
         Me.BTT_TRAY_SILENT_MODE = New System.Windows.Forms.ToolStripMenuItem()
         Me.BTT_TRAY_FEED_SHOW = New System.Windows.Forms.ToolStripMenuItem()
+        Me.BTT_TRAY_CHANNELS = New System.Windows.Forms.ToolStripMenuItem()
         Me.BTT_TRAY_SHOW_HIDE = New System.Windows.Forms.ToolStripMenuItem()
         Me.BTT_TRAY_CLOSE = New System.Windows.Forms.ToolStripMenuItem()
         Me.BTT_TRAY_CLOSE_NO_SCRIPT = New System.Windows.Forms.ToolStripMenuItem()
@@ -418,7 +419,7 @@ Partial Public Class MainFrame : Inherits System.Windows.Forms.Form
         'BTT_DOWN_VIDEO
         '
         Me.BTT_DOWN_VIDEO.AutoToolTip = True
-        Me.BTT_DOWN_VIDEO.DisplayStyle = System.Windows.Forms.ToolStripItemDisplayStyle.Text
+        Me.BTT_DOWN_VIDEO.Image = Global.SCrawler.My.Resources.Resources.ArrowDownPic_Blue_24
         Me.BTT_DOWN_VIDEO.ImageTransparentColor = System.Drawing.Color.Magenta
         Me.BTT_DOWN_VIDEO.Name = "BTT_DOWN_VIDEO"
         Me.BTT_DOWN_VIDEO.Size = New System.Drawing.Size(231, 22)
@@ -754,6 +755,7 @@ Partial Public Class MainFrame : Inherits System.Windows.Forms.Form
         '
         'BTT_CONTEXT_GROUPS
         '
+        Me.BTT_CONTEXT_GROUPS.Image = Global.SCrawler.My.Resources.Resources.TagPic_24
         Me.BTT_CONTEXT_GROUPS.Name = "BTT_CONTEXT_GROUPS"
         Me.BTT_CONTEXT_GROUPS.Size = New System.Drawing.Size(221, 22)
         Me.BTT_CONTEXT_GROUPS.Text = "Change labels"
@@ -824,9 +826,9 @@ Partial Public Class MainFrame : Inherits System.Windows.Forms.Form
         '
         'TRAY_CONTEXT
         '
-        Me.TRAY_CONTEXT.Items.AddRange(New System.Windows.Forms.ToolStripItem() {Me.BTT_TRAY_PAUSE_AUTOMATION, Me.BTT_TRAY_SILENT_MODE, Me.BTT_TRAY_FEED_SHOW, TRAY_SEP_1, Me.BTT_TRAY_SHOW_HIDE, TRAY_SEP_2, Me.BTT_TRAY_CLOSE, Me.BTT_TRAY_CLOSE_NO_SCRIPT})
+        Me.TRAY_CONTEXT.Items.AddRange(New System.Windows.Forms.ToolStripItem() {Me.BTT_TRAY_PAUSE_AUTOMATION, Me.BTT_TRAY_SILENT_MODE, Me.BTT_TRAY_FEED_SHOW, Me.BTT_TRAY_CHANNELS, TRAY_SEP_1, Me.BTT_TRAY_SHOW_HIDE, TRAY_SEP_2, Me.BTT_TRAY_CLOSE, Me.BTT_TRAY_CLOSE_NO_SCRIPT})
         Me.TRAY_CONTEXT.Name = "TRAY_CONTEXT"
-        Me.TRAY_CONTEXT.Size = New System.Drawing.Size(171, 148)
+        Me.TRAY_CONTEXT.Size = New System.Drawing.Size(171, 170)
         '
         'BTT_TRAY_PAUSE_AUTOMATION
         '
@@ -854,6 +856,13 @@ Partial Public Class MainFrame : Inherits System.Windows.Forms.Form
         Me.BTT_TRAY_FEED_SHOW.Text = "Feed"
         Me.BTT_TRAY_FEED_SHOW.ToolTipText = "Show feed of recently downloaded data." & Global.Microsoft.VisualBasic.ChrW(13) & Global.Microsoft.VisualBasic.ChrW(10) & "Ctrl+Click the tray icon to show the feed" &
     "."
+        '
+        'BTT_TRAY_CHANNELS
+        '
+        Me.BTT_TRAY_CHANNELS.Image = Global.SCrawler.My.Resources.SiteResources.RedditPic_512
+        Me.BTT_TRAY_CHANNELS.Name = "BTT_TRAY_CHANNELS"
+        Me.BTT_TRAY_CHANNELS.Size = New System.Drawing.Size(170, 22)
+        Me.BTT_TRAY_CHANNELS.Text = "Channels"
         '
         'BTT_TRAY_SHOW_HIDE
         '
@@ -986,4 +995,5 @@ Partial Public Class MainFrame : Inherits System.Windows.Forms.Form
     Friend WithEvents BTT_DOWN_AUTOMATION_PAUSE As ToolStripMenuItem
     Private WithEvents BTT_TRAY_FEED_SHOW As ToolStripMenuItem
     Friend WithEvents MENU_DOWN_ALL As ToolStripDropDownButton
+    Private WithEvents BTT_TRAY_CHANNELS As ToolStripMenuItem
 End Class

SCrawler/MainFrame.vb

@@ -88,32 +88,33 @@ Public Class MainFrame
             LIST_PROFILES.ShowGroups = .UseGrouping
             ApplyViewPattern(.ViewMode.Value)
             AddHandler .Labels.NewLabelAdded, AddressOf UpdateLabelsGroups
+            UserListLoader = New ListImagesLoader(LIST_PROFILES)
+            RefillList()
+            UpdateLabelsGroups()
+            SetShowButtonsCheckers(.ShowingMode.Value)
+            CheckVersion(False)
+            BTT_SITE_ALL.Checked = .SelectedSites.Count = 0
+            BTT_SITE_SPECIFIC.Checked = .SelectedSites.Count > 0
+            BTT_SHOW_LIMIT_DATES_NOT.Tag = ShowingDates.Not
+            BTT_SHOW_LIMIT_DATES_NOT.Checked = .ViewDateMode.Value = ShowingDates.Not
+            BTT_SHOW_LIMIT_DATES_IN.Tag = ShowingDates.In
+            BTT_SHOW_LIMIT_DATES_IN.Checked = .ViewDateMode.Value = ShowingDates.In
+            With .Groups
+                AddHandler .Added, AddressOf GROUPS_Added
+                AddHandler .Deleted, AddressOf GROUPS_Deleted
+                AddHandler .Updated, AddressOf GROUPS_Updated
+                If .Count > 0 Then
+                    For Each ugroup As Groups.DownloadGroup In Settings.Groups : GROUPS_Added(ugroup) : Next
+                End If
+            End With
+            .Automation = New Scheduler
+            AddHandler .Groups.Updated, AddressOf .Automation.GROUPS_Updated
+            AddHandler .Groups.Deleted, AddressOf .Automation.GROUPS_Deleted
+            AddHandler .Automation.PauseDisabled, AddressOf MainFrameObj.PauseButtons.UpdatePauseButtons
+            If .Automation.Count > 0 Then .Labels.AddRange(.Automation.GetGroupsLabels, False) : .Labels.Update()
+            _UFinit = False
+            Await .Automation.Start(True)
         End With
-        UserListLoader = New ListImagesLoader(LIST_PROFILES)
-        RefillList()
-        UpdateLabelsGroups()
-        SetShowButtonsCheckers(Settings.ShowingMode.Value)
-        CheckVersion(False)
-        BTT_SITE_ALL.Checked = Settings.SelectedSites.Count = 0
-        BTT_SITE_SPECIFIC.Checked = Settings.SelectedSites.Count > 0
-        BTT_SHOW_LIMIT_DATES_NOT.Tag = ShowingDates.Not
-        BTT_SHOW_LIMIT_DATES_NOT.Checked = Settings.ViewDateMode.Value = ShowingDates.Not
-        BTT_SHOW_LIMIT_DATES_IN.Tag = ShowingDates.In
-        BTT_SHOW_LIMIT_DATES_IN.Checked = Settings.ViewDateMode.Value = ShowingDates.In
-        With Settings.Groups
-            AddHandler .Added, AddressOf GROUPS_Added
-            AddHandler .Deleted, AddressOf GROUPS_Deleted
-            AddHandler .Updated, AddressOf GROUPS_Updated
-            If .Count > 0 Then
-                For Each ugroup As Groups.DownloadGroup In Settings.Groups : GROUPS_Added(ugroup) : Next
-            End If
-        End With
-        Settings.Automation = New Scheduler
-        AddHandler Settings.Groups.Updated, AddressOf Settings.Automation.GROUPS_Updated
-        AddHandler Settings.Groups.Deleted, AddressOf Settings.Automation.GROUPS_Deleted
-        AddHandler Settings.Automation.PauseDisabled, AddressOf MainFrameObj.PauseButtons.UpdatePauseButtons
-        _UFinit = False
-        Await Settings.Automation.Start(True)
         UpdatePauseButtonsVisibility()
         GoTo EndFunction
 FormClosingInvoker:
@@ -250,7 +251,6 @@ CloseResume:
 #Region "List refill, update"
     Friend Sub RefillList()
         UserListLoader.Update()
-        GC.Collect()
     End Sub
     Private Sub UserListUpdate(ByVal User As IUserData, ByVal Add As Boolean)
         UserListLoader.UpdateUser(User, Add)
@@ -380,7 +380,7 @@ CloseResume:
         If MyFeed Is Nothing Then MyFeed = New DownloadFeedForm : AddHandler Downloader.FeedFilesChanged, AddressOf MyFeed.Downloader_FilesChanged
         If MyFeed.Visible Then MyFeed.BringToFront() Else MyFeed.Show()
     End Sub
-    Private Sub BTT_CHANNELS_Click(sender As Object, e As EventArgs) Handles BTT_CHANNELS.Click
+    Private Sub BTT_CHANNELS_Click(sender As Object, e As EventArgs) Handles BTT_CHANNELS.Click, BTT_TRAY_CHANNELS.Click
         If MyChannels Is Nothing Then
             MyChannels = New ChannelViewForm
             AddHandler MyChannels.OnUsersAdded, AddressOf OnUsersAddedHandler
@@ -1396,6 +1396,7 @@ ResumeDownloadingOperation:
                 If result < 6 Then
                     Dim collectionResult% = -1
                     Dim tmpResult%
+                    Dim tmpUserNames As New List(Of String)
                     Dim IsMultiple As Boolean = users.Count > 1
                     Dim removedUsers As New List(Of String)
                     Dim keepData As Boolean = Not (result Mod 2) = 0
@@ -1430,10 +1431,18 @@ ResumeDownloadingOperation:
                             removedUsers.Add(ugn(user))
                             user.Dispose()
                         Else
+                            If banUser Then
+                                tmpUserNames.Clear()
+                                If user.IsCollection Then
+                                    tmpUserNames.ListAddList(DirectCast(user, UserDataBind).Collections.Select(Function(u) u.Name), l)
+                                Else
+                                    tmpUserNames.Add(user.Name)
+                                End If
+                            End If
                             tmpResult = user.Delete(IsMultiple, collectionResult)
                             If user.IsCollection And collectionResult = -1 Then collectionResult = tmpResult
                             If tmpResult > 0 Then
-                                If banUser Then Settings.BlackList.ListAddValue(New UserBan(user.Name, reason), l) : b = True
+                                If banUser And tmpUserNames.Count > 0 Then Settings.BlackList.ListAddList(tmpUserNames.Select(Function(u) New UserBan(u, reason)), l) : b = True
                                 RemoveUserFromList(user)
                                 removedUsers.Add(ugn(user))
                             Else
@@ -1452,8 +1461,8 @@ ResumeDownloadingOperation:
                         m.Text = "No one user deleted!"
                         m.Style = MsgBoxStyle.Critical
                     Else
-                        m.Text = $"The following users were deleted:{vbNewLine}{removedUsers.ListToStringE(vbNewLine, userProvider)}{vbNewLine.StringDup(2)}"
+                        m.Text = $"The following users were deleted:{vbNewLine}{removedUsers.ListToString(vbNewLine)}{vbNewLine.StringDup(2)}"
-                        m.Text &= $"The following users were NOT deleted:{vbNewLine}{leftUsers.ListToStringE(vbNewLine, userProvider)}"
+                        m.Text &= $"The following users were NOT deleted:{vbNewLine}{leftUsers.ListToString(vbNewLine)}"
                         m.Style = MsgBoxStyle.Exclamation
                     End If
                     If b Then Settings.UpdateBlackList()

SCrawler/MainFrameObjects.vb

@@ -74,14 +74,18 @@ Friend Class MainFrameObjects
     End Sub
     Private Sub Notificator_OnClicked(ByVal Key As String) Handles Notificator.OnClicked
         If Not Key.IsEmptyString Then
+            Dim found As Boolean = False
+            Dim activateForm As Boolean = False
             If Key.StartsWith(NotificationInternalKey) Then
                 Select Case Key
                     Case $"{NotificationInternalKey}_{NotifyObj.Channels}" : MF.MyChannels.FormShowS()
                     Case $"{NotificationInternalKey}_{NotifyObj.SavedPosts}" : MF.MySavedPosts.FormShowS()
                     Case Else : Focus(True)
                 End Select
-            ElseIf Settings.Automation Is Nothing OrElse Not Settings.Automation.NotificationClicked(Key) Then
+            ElseIf Settings.Automation Is Nothing OrElse Not Settings.Automation.NotificationClicked(Key, found, activateForm) Then
                 Focus(True)
+            ElseIf found Then
+                Focus(activateForm)
             Else
                 Focus(True)
             End If

SCrawler/MainMod.vb

@@ -6,6 +6,7 @@
 '
 ' This program is distributed in the hope that it will be useful,
 ' but WITHOUT ANY WARRANTY
+Imports System.Runtime.CompilerServices
 Imports PersonalUtilities.Functions.XML.Base
 Imports PersonalUtilities.Functions.RegularExpressions
 Imports PersonalUtilities.Forms.Toolbars
@@ -147,6 +148,14 @@ Friend Module MainMod
             Return $"{If(Host?.Name, String.Empty)}{Opt}"
         End If
     End Function
+    <Extension> Friend Function GetGroupsLabels(Of T As Groups.IGroup)(ByVal Groups As IEnumerable(Of T)) As List(Of String)
+        If Groups.ListExists Then
+            Return ListAddList(Nothing, Groups.SelectMany(Function(g) g.Labels), LAP.NotContainsOnly).
+                   ListAddList(Groups.SelectMany(Function(g) g.LabelsExcluded), LAP.NotContainsOnly)
+        Else
+            Return Nothing
+        End If
+    End Function
 #Region "Standalone video download functions"
     Friend Function GetCurrentBuffer() As String
         Dim b$ = BufferText

SCrawler/My Project/AssemblyInfo.vb

@@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices
 ' by using the '*' as shown below:
 ' <Assembly: AssemblyVersion("1.0.*")>
 
-<Assembly: AssemblyVersion("2022.11.16.0")>
+<Assembly: AssemblyVersion("2023.1.27.0")>
-<Assembly: AssemblyFileVersion("2022.11.16.0")>
+<Assembly: AssemblyFileVersion("2023.1.27.0")>
 <Assembly: NeutralResourcesLanguage("en")>

SCrawler/My Project/Resources.Designer.vb

@@ -370,6 +370,16 @@ Namespace My.Resources
             End Get
         End Property
         
+        '''<summary>
+        '''  Looks up a localized resource of type System.Drawing.Bitmap.
+        '''</summary>
+        Friend ReadOnly Property TagPic_24() As System.Drawing.Bitmap
+            Get
+                Dim obj As Object = ResourceManager.GetObject("TagPic_24", resourceCulture)
+                Return CType(obj,System.Drawing.Bitmap)
+            End Get
+        End Property
+        
         '''<summary>
         '''  Looks up a localized resource of type System.Drawing.Icon similar to (Icon).
         '''</summary>

SCrawler/My Project/Resources.resx

@@ -214,4 +214,7 @@
   <data name="TagIcon_32" type="System.Resources.ResXFileRef, System.Windows.Forms">
     <value>..\Content\Icons\TagIcon_32.ico;System.Drawing.Icon, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a</value>
   </data>
+  <data name="TagPic_24" type="System.Resources.ResXFileRef, System.Windows.Forms">
+    <value>..\Content\Pictures\TagPic_24.png;System.Drawing.Bitmap, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a</value>
+  </data>
 </root>

SCrawler/PluginsEnvironment/Hosts/PropertyValueHost.vb

@@ -76,6 +76,7 @@ Namespace Plugin.Hosts
                                 .EndInit(True)
                             End With
                             AddHandler .ActionOnButtonClick, AddressOf TextBoxClick
+                            If Not ProviderValue Is Nothing AndAlso ProviderValueInteraction Then AddHandler .ActionOnTextChanged, AddressOf TextBoxTextChanged
                         End With
                     End If
                 End If
@@ -98,6 +99,14 @@ Namespace Plugin.Hosts
                 ErrorsDescriber.Execute(EDP.LogMessageValue, ex, $"Updating [{Name}] property")
             End Try
         End Sub
+        Private Sub TextBoxTextChanged(ByVal Sender As Object, ByVal e As EventArgs)
+            UpdateProviderPropertyName()
+            With DirectCast(Sender, TextBoxExtended)
+                Dim s% = .SelectionStart
+                Dim t$ = AConvert(Of String)(.Text, ProviderValue, String.Empty)
+                If Not t = .Text Then .Text = t : .Select(s, 0)
+            End With
+        End Sub
         Friend Sub UpdateValueByControl()
             If Not Control Is Nothing AndAlso Not TypeOf Control Is Label Then
                 If TypeOf Control Is CheckBox Then
@@ -105,6 +114,7 @@ Namespace Plugin.Hosts
                         If Options.ThreeStates Then Value = CInt(.CheckState) Else Value = .Checked
                     End With
                 Else
+                    UpdateProviderPropertyName()
                     Value = AConvert(DirectCast(Control, TextBoxExtended).Text, AModes.Var, [Type],,,, ProviderValue)
                 End If
             End If
@@ -116,12 +126,16 @@ Namespace Plugin.Hosts
                         If Options.ThreeStates Then Return CInt(.CheckState) Else Return .Checked
                     End With
                 Else
+                    UpdateProviderPropertyName()
                     Return AConvert(DirectCast(Control, TextBoxExtended).Text, AModes.Var, [Type],,,, ProviderValue)
                 End If
             Else
                 Return Nothing
             End If
         End Function
+        Private Sub UpdateProviderPropertyName()
+            If ProviderValueIsPropertyProvider Then DirectCast(ProviderValue, IPropertyProvider).PropertyName = Name
+        End Sub
 #End Region
 #Region "Compatibility"
         Private ReadOnly Source As Object
@@ -143,9 +157,17 @@ Namespace Plugin.Hosts
         End Property
 #Region "Providers"
         Friend Property ProviderFieldsChecker As IFormatProvider
-        Friend Property ProviderValue As IFormatProvider
+        Private Property ProviderValue As IFormatProvider
-        Friend Sub SetProvider(ByVal Provider As IFormatProvider, ByVal FC As Boolean)
+        Private Property ProviderValueInteraction As Boolean = False
-            If FC Then ProviderFieldsChecker = Provider Else ProviderValue = Provider
+        Private Property ProviderValueIsPropertyProvider As Boolean = False
+        Friend Sub SetProvider(ByVal Provider As IFormatProvider, ByVal Instance As Provider)
+            If Instance.FieldsChecker Then
+                ProviderFieldsChecker = Provider
+            Else
+                ProviderValue = Provider
+                ProviderValueIsPropertyProvider = TypeOf ProviderValue Is IPropertyProvider
+                ProviderValueInteraction = Instance.Interaction
+            End If
         End Sub
 #End Region
         Friend PropertiesChecking As String()

SCrawler/PluginsEnvironment/Hosts/SettingsHost.vb

@@ -80,7 +80,7 @@ Namespace Plugin.Hosts
         Friend ReadOnly Property HasSpecialOptions As Boolean = False
         Private ReadOnly _ResponserGetMethod As MethodInfo
         Private ReadOnly _ResponserIsContainer As Boolean = False
-        Friend ReadOnly Property Responser As Response
+        Friend ReadOnly Property Responser As Responser
             Get
                 If Not _ResponserGetMethod Is Nothing Then
                     Return _ResponserGetMethod.Invoke(Source, Nothing)
@@ -190,7 +190,7 @@ Namespace Plugin.Hosts
                     If m.MemberType = MemberTypes.Property Then
                         PropList.Add(New PropertyValueHost(Source, m))
                         With DirectCast(m, PropertyInfo)
-                            If .PropertyType Is GetType(Response) AndAlso m.GetCustomAttribute(Of DoNotUse)() Is Nothing Then _ResponserGetMethod = .GetMethod
+                            If .PropertyType Is GetType(Responser) AndAlso m.GetCustomAttribute(Of DoNotUse)() Is Nothing Then _ResponserGetMethod = .GetMethod
                         End With
                     End If
                     With m.GetCustomAttributes()
@@ -204,7 +204,7 @@ Namespace Plugin.Hosts
                                     End If
                                 Next
                             ElseIf m.MemberType = MemberTypes.Property Then
-                                If Not m.GetCustomAttribute(Of Provider)() Is Nothing Then Providers.Add(m)
+                                If m.GetCustomAttributes(Of Provider)().ListExists Then Providers.Add(m)
                             End If
                         End If
                     End With
@@ -220,13 +220,15 @@ Namespace Plugin.Hosts
                     Updaters.Clear()
                 End If
                 If Providers.Count > 0 Then
-                    Dim prov As Provider
+                    Dim prov As IEnumerable(Of Provider)
+                    Dim _prov As Provider
                     For Each m In Providers
-                        prov = m.GetCustomAttribute(Of Provider)()
+                        prov = m.GetCustomAttributes(Of Provider)()
-                        i = PropList.FindIndex(Function(p) p.Name = prov.Name)
+                        If prov.ListExists Then
-                        If i >= 0 Then
+                            For Each _prov In prov
-                            PropList(i).SetProvider(DirectCast(DirectCast(m, PropertyInfo).GetValue(Source), IFormatProvider),
+                                i = PropList.FindIndex(Function(p) p.Name = _prov.Name)
-                                                    m.GetCustomAttribute(Of Provider)().FieldsChecker)
+                                If i >= 0 Then PropList(i).SetProvider(DirectCast(DirectCast(m, PropertyInfo).GetValue(Source), IFormatProvider), _prov)
+                            Next
                         End If
                     Next
                     Providers.Clear()

SCrawler/SCrawler.vbproj

@@ -165,12 +165,6 @@
     <Compile Include="API\Base\ProfileSaved.vb" />
     <Compile Include="API\Base\SiteSettingsBase.vb" />
     <Compile Include="API\Base\Structures.vb" />
-    <Compile Include="API\Instagram\AdditionalSettingsForm.Designer.vb">
-      <DependentUpon>AdditionalSettingsForm.vb</DependentUpon>
-    </Compile>
-    <Compile Include="API\Instagram\AdditionalSettingsForm.vb">
-      <SubType>Form</SubType>
-    </Compile>
     <Compile Include="API\Instagram\EditorExchangeOptions.vb" />
     <Compile Include="API\Instagram\OptionsForm.Designer.vb">
       <DependentUpon>OptionsForm.vb</DependentUpon>
@@ -178,7 +172,6 @@
     <Compile Include="API\Instagram\OptionsForm.vb">
       <SubType>Form</SubType>
     </Compile>
-    <Compile Include="API\Instagram\SettingsExchangeOptions.vb" />
     <Compile Include="API\LPSG\Declarations.vb" />
     <Compile Include="API\LPSG\SiteSettings.vb" />
     <Compile Include="API\LPSG\UserData.vb" />
@@ -203,6 +196,13 @@
     <Compile Include="API\TikTok\Declarations.vb" />
     <Compile Include="API\TikTok\SiteSettings.vb" />
     <Compile Include="API\TikTok\UserData.vb" />
+    <Compile Include="API\Twitter\EditorExchangeOptions.vb" />
+    <Compile Include="API\Twitter\OptionsForm.Designer.vb">
+      <DependentUpon>OptionsForm.vb</DependentUpon>
+    </Compile>
+    <Compile Include="API\Twitter\OptionsForm.vb">
+      <SubType>Form</SubType>
+    </Compile>
     <Compile Include="API\Xhamster\Declarations.vb" />
     <Compile Include="API\Xhamster\M3U8.vb" />
     <Compile Include="API\Xhamster\SiteSettings.vb" />
@@ -414,9 +414,6 @@
     </Compile>
   </ItemGroup>
   <ItemGroup>
-    <EmbeddedResource Include="API\Instagram\AdditionalSettingsForm.resx">
-      <DependentUpon>AdditionalSettingsForm.vb</DependentUpon>
-    </EmbeddedResource>
     <EmbeddedResource Include="API\Instagram\OptionsForm.resx">
       <DependentUpon>OptionsForm.vb</DependentUpon>
     </EmbeddedResource>
@@ -426,6 +423,9 @@
     <EmbeddedResource Include="API\Reddit\RedditViewSettingsForm.resx">
       <DependentUpon>RedditViewSettingsForm.vb</DependentUpon>
     </EmbeddedResource>
+    <EmbeddedResource Include="API\Twitter\OptionsForm.resx">
+      <DependentUpon>OptionsForm.vb</DependentUpon>
+    </EmbeddedResource>
     <EmbeddedResource Include="Channels\ChannelsStatsForm.resx">
       <DependentUpon>ChannelsStatsForm.vb</DependentUpon>
     </EmbeddedResource>
@@ -584,6 +584,7 @@
     <None Include="Content\Icons\SiteIcons\XhamsterIcon_32.ico" />
     <None Include="Content\Pictures\SitePictures\PornHubPic_16.png" />
     <None Include="Content\Icons\SiteIcons\PornHubIcon_16.ico" />
+    <None Include="Content\Pictures\TagPic_24.png" />
     <Content Include="cURL\curl.exe">
       <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
     </Content>

SCrawler/SettingsCLS.vb

@@ -23,12 +23,11 @@ Friend Class SettingsCLS : Implements IDisposable
     Friend Const CookieEncryptKey As String = "SCrawlerCookiesEncryptKeyword"
     Friend ReadOnly Design As XmlFile
     Private ReadOnly MyXML As XmlFile
-    Friend ReadOnly OS64 As Boolean
     Private ReadOnly FfmpegExists As Boolean
     Friend ReadOnly FfmpegFile As SFile
     Friend ReadOnly Property UseM3U8 As Boolean
         Get
-            Return OS64 And FfmpegExists
+            Return FfmpegExists
         End Get
     End Property
     Private ReadOnly FFMPEGNotification As XMLValue(Of Boolean)
@@ -67,7 +66,6 @@ Friend Class SettingsCLS : Implements IDisposable
     End Sub
     Friend Sub New()
         RemoveUnusedPlugins()
-        OS64 = Environment.Is64BitOperatingSystem
         FfmpegFile = "ffmpeg.exe"
         FfmpegExists = FfmpegFile.Exists
 
@@ -81,7 +79,7 @@ Friend Class SettingsCLS : Implements IDisposable
         LastCollections = New List(Of String)
 
         FFMPEGNotification = New XMLValue(Of Boolean)("FFMPEGNotification", True, MyXML)
-        If OS64 And Not FfmpegExists Then
+        If Not FfmpegExists Then
             If FFMPEGNotification.Value AndAlso MsgBoxE(New MMessage("[ffmpeg.exe] is missing", "ffmpeg.exe",
                                                         {"OK", New MsgBoxButton("Disable notification") With {
                                                         .IsDialogResultButton = False, .ToolTip = "Disable ffmpeg missing notification"}}, vbExclamation) With {
@@ -121,7 +119,7 @@ Friend Class SettingsCLS : Implements IDisposable
         If tmpPluginList.ListExists Then Plugins.AddRange(tmpPluginList)
         CookiesEncrypted.Value = True
 
-        FastProfilesLoading = New XMLValue(Of Boolean)("FastProfilesLoading", False, MyXML)
+        FastProfilesLoading = New XMLValue(Of Boolean)("FastProfilesLoading", True, MyXML)
         MaxLargeImageHeight = New XMLValue(Of Integer)("MaxLargeImageHeight", 150, MyXML)
         MaxSmallImageHeight = New XMLValue(Of Integer)("MaxSmallImageHeight", 15, MyXML)
         DownloadOpenInfo = New XMLValueAttribute(Of Boolean, Boolean)("DownloadOpenInfo", "OpenAgain", False, False, MyXML)
@@ -208,7 +206,7 @@ Friend Class SettingsCLS : Implements IDisposable
 
         Labels = New LabelsKeeper(MyXML)
         Groups = New Groups.DownloadGroupCollection
-        Labels.AddRange(Groups.GetLabels, False)
+        Labels.AddRange(Groups.GetGroupsLabels, False)
 
         MyXML.EndUpdate()
         If MyXML.ChangesDetected Then MyXML.Sort() : MyXML.UpdateData()
@@ -259,7 +257,7 @@ Friend Class SettingsCLS : Implements IDisposable
                     If cUsers.ListExists Then
                         Dim d As New Dictionary(Of String, List(Of UserInfo))
                         cUsers = cUsers.ListForEachCopy(Of List(Of UserInfo))(Function(ByVal f As UserInfo, ByVal f_indx As Integer) As UserInfo
-                                                                                  Dim m% = IIf(f.Merged Or f.IsVirual, 1, 2)
+                                                                                  Dim m% = IIf(f.Merged Or f.IsVirtual, 1, 2)
                                                                                   If Not f.Protected AndAlso SFile.GetPath(f.File.CutPath(m - 1).Path).Exists(SFO.Path, False) Then
                                                                                       If Not d.ContainsKey(f.CollectionName) Then
                                                                                           d.Add(f.CollectionName, New List(Of UserInfo) From {f})
@@ -319,11 +317,8 @@ Friend Class SettingsCLS : Implements IDisposable
                 If NeedUpdate Then UpdateUsersList()
             End If
             If Users.Count > 0 Then
-                Dim tul As IEnumerable(Of String) = Users.SelectMany(Function(u) u.Labels)
+                Labels.AddRange(Users.SelectMany(Function(u) u.Labels), False)
-                Labels.AddRange(tul, False)
+                Labels.Update()
-                If Labels.NewLabelsExists Or
-                   (tul.ListExists AndAlso Not tul.Contains(LabelsKeeper.NoParsedUser) AndAlso Labels.Remove(LabelsKeeper.NoParsedUser)) Then _
-                   Labels.Update() : Labels.NewLabels.Clear() : Labels.Verify()
             End If
         Catch ex As Exception
         End Try

SCrawler/UserFinder.vb

@@ -86,6 +86,7 @@ Friend Class UserFinder : Implements IDisposable
                                 .CollectionName = x.Value(UserInfo.Name_Collection),
                                 .IsChannel = x.Value(UserInfo.Name_IsChannel).FromXML(Of Boolean)(False)
                             }
+                            'TODELETE: UserFinder remove old 'merge' constant
 #Disable Warning BC40000
                             If x.Contains(UserDataBase.Name_DataMerging) Then
                                 u.Merged = x.Value(UserDataBase.Name_DataMerging).FromXML(Of Boolean)(False)
@@ -158,7 +159,7 @@ Friend Class UserFinder : Implements IDisposable
         Const MsgTitle$ = "Import users"
         Const DesignNode$ = "ImportUserSelector"
         Try
-            Dim uStr As Func(Of UserInfo, String) = Function(u) $"{IIf(u.CollectionName.IsEmptyString, String.Empty, $"[{u.CollectionName}]: ")} {u.Site} - {u.Name}"
+            Dim uStr As Func(Of UserInfo, String) = Function(u) $"{IIf(u.CollectionName.IsEmptyString, String.Empty, $"[{u.CollectionName}]: ")}{u.Site} - {u.Name}"
             Dim uc As Comparison(Of UserInfo) = Function(ByVal x As UserInfo, ByVal y As UserInfo) As Integer
                                                     If Not x.CollectionName.IsEmptyString And Not y.CollectionName.IsEmptyString Then
                                                         Return x.CollectionName.CompareTo(y.CollectionName)
@@ -182,7 +183,7 @@ Friend Class UserFinder : Implements IDisposable
             End If
             __added = {__added, __dup, __skipped}.ListToString(vbCr.StringDup(2))
             If Not __added.IsEmptyString Then
-                Using t As New TextSaver($"LOGs\ImportUsers.txt") With {.ForceAddDateTimeToFileName = True}
+                Using t As New TextSaver("LOGs\ImportUsers.txt") With {.ForceAddDateTimeToFileName = True}
                     t.Append(__added)
                     If Added.Count > 0 Then
                         t.AppendLine(vbNewLine.StringDup(2))
@@ -296,7 +297,7 @@ Friend Class UserFinder : Implements IDisposable
                 AddHandler f.AddClick, __add
                 If f.ShowDialog() = DialogResult.OK Then
                     l.Clear()
-                    l.AddRange(f.DataResult)
+                    l.ListAddList(f.DataResult, LAP.NotContainsOnly)
                     Return l
                 End If
             End Using

SCrawler/UserInfo.vb

@@ -39,7 +39,7 @@ Partial Friend Module MainMod
                 Return Not CollectionName.IsEmptyString
             End Get
         End Property
-        Friend ReadOnly Property IsVirual As Boolean
+        Friend ReadOnly Property IsVirtual As Boolean
             Get
                 Return CollectionModel = UsageModel.Virtual Or UserModel = UsageModel.Virtual
             End Get
@@ -123,7 +123,7 @@ Partial Friend Module MainMod
             ElseIf Merged And IncludedInCollection Then
                 Return $"{ColPath}\{SettingsFolderName}"
             Else
-                If IncludedInCollection And Not IsVirual Then
+                If IncludedInCollection And Not IsVirtual Then
                     Return $"{ColPath}\{Site}_{Name}\{SettingsFolderName}"
                 ElseIf Not Settings(Plugin) Is Nothing Then
                     Return $"{Settings(Plugin).Path.PathNoSeparator}\{Name}\{SettingsFolderName}"

SCrawler/UserSearchForm.vb

@@ -94,56 +94,49 @@ Friend Class UserSearchForm
             LIST_SEARCH.BeginUpdate()
             ControlInvokeFast(LIST_SEARCH, Sub() LIST_SEARCH.Items.Clear())
             Results.Clear()
-            Dim t$ = TXT_SEARCH.Text.Trim
+            Dim t$ = TXT_SEARCH.Text.StringTrim.StringToLower
             With Settings
                 If Not t.IsEmptyString And .Users.Count > 0 Then
                     Dim i%
                     Dim s As Plugin.ExchangeOptions = Nothing
-                    Dim cu As Boolean = False
                     Dim __descr As Boolean = CH_SEARCH_IN_DESCR.Checked
                     Dim __name As Boolean = CH_SEARCH_IN_NAME.Checked
                     Dim __lbl As Boolean = CH_SEARCH_IN_LABEL.Checked
-                    Dim _CheckUrl As Action(Of IUserData) = Sub(ByVal u As IUserData)
+                    Dim __isUrl As Boolean = t.StartsWith("http")
-                                                                If cu AndAlso ((u.Site = s.SiteName Or u.HOST.Key = s.HostKey) And u.Name.ToLower = s.UserName) Then _
+                    Dim __urlFound As Boolean = False
-                                                                   Results.ListAddValue(New SearchResult(u, SearchResult.Modes.URL), RLP)
+                    Dim _p_url As Predicate(Of IUserData) = Function(u) __urlFound AndAlso ((u.Site = s.SiteName Or u.HOST.Key = s.HostKey) And u.Name.ToLower = s.UserName.ToLower)
-                                                            End Sub
+                    Dim _p_descr As Predicate(Of IUserData) = Function(u) __descr AndAlso Not u.Description.IsEmptyString AndAlso u.Description.ToLower.Contains(t)
-                    Dim _CheckDescr As Action(Of IUserData) = Sub(ByVal u As IUserData)
+                    Dim _p_labels_p As Predicate(Of String) = Function(l) l.ToLower.Contains(t)
-                                                                  If __descr AndAlso Not u.Description.IsEmptyString AndAlso
+                    Dim _p_labels As Predicate(Of IUserData) = Function(u) __lbl AndAlso u.Labels.ListExists AndAlso u.Labels.Exists(_p_labels_p)
-                                                                     u.Description.Contains(t) Then _
+                    Dim _addValue As Action(Of IUserData, SearchResult.Modes, Predicate(Of IUserData)) = Sub(u, m, p) If p.Invoke(u) Then Results.ListAddValue(New SearchResult(u, m), RLP)
-                                                                     Results.ListAddValue(New SearchResult(u, SearchResult.Modes.Description), RLP)
-                                                              End Sub
-                    Dim _LabelPredicate As Predicate(Of String) = Function(l) l.ToLower.Contains(t)
-                    Dim _CheckLabels As Action(Of IUserData) = Sub(ByVal u As IUserData)
-                                                                   If __lbl AndAlso u.Labels.ListExists AndAlso u.Labels.Exists(_LabelPredicate) Then _
-                                                                      Results.ListAddValue(New SearchResult(u, SearchResult.Modes.Label), RLP)
-                                                               End Sub
 
-                    If t.Length >= 4 AndAlso t.StartsWith("http") Then
+                    If __isUrl Then
                         For Each p In Settings.Plugins
                             s = p.Settings.IsMyUser(t)
                             If Not s.UserName.IsEmptyString Then Exit For
                         Next
+                        __urlFound = Not s.UserName.IsEmptyString
                     End If
-                    cu = Not s.UserName.IsEmptyString
-                    t = t.ToLower
 
                     For Each user As IUserData In .Users
-                        If __name AndAlso user.Name.ToLower.Contains(t) Then Results.ListAddValue(New SearchResult(user, SearchResult.Modes.Name), RLP)
+                        If Not __isUrl AndAlso __name AndAlso user.Name.ToLower.Contains(t) Then Results.ListAddValue(New SearchResult(user, SearchResult.Modes.Name), RLP)
                         If user.IsCollection Then
                             With DirectCast(user, UserDataBind)
                                 If .Count > 0 Then
                                     For i = 0 To .Count - 1
-                                        If __name AndAlso .Item(i).Name.ToLower = t Then Results.ListAddValue(New SearchResult(.Item(i), SearchResult.Modes.Name), RLP)
+                                        With .Item(i)
-                                        _CheckUrl(.Item(i))
+                                            If Not __isUrl AndAlso __name AndAlso .Self.Name.ToLower = t Then Results.ListAddValue(New SearchResult(.Self, SearchResult.Modes.Name), RLP)
-                                        _CheckDescr(.Item(i))
+                                            _addValue(.Self, SearchResult.Modes.URL, _p_url)
-                                        _CheckLabels(.Item(i))
+                                            _addValue(.Self, SearchResult.Modes.Description, _p_descr)
+                                            _addValue(.Self, SearchResult.Modes.Label, _p_labels)
+                                        End With
                                     Next
                                 End If
                             End With
                         Else
-                            _CheckUrl(user)
+                            _addValue(user, SearchResult.Modes.URL, _p_url)
-                            _CheckDescr(user)
+                            _addValue(user, SearchResult.Modes.Description, _p_descr)
-                            _CheckLabels(user)
+                            _addValue(user, SearchResult.Modes.Label, _p_labels)
                         End If
                     Next
                     If Results.Count > 0 Then