diff --git a/Changelog.md b/Changelog.md index 3fe735c..530d90b 100644 --- a/Changelog.md +++ b/Changelog.md @@ -1,3 +1,14 @@ +# 2022.9.8.0 + +*2022-09-08* + +- Added + - **Feed** (feed of downloaded media files) + - Missing posts tracking and management + - Simple scheduler notifications +- Fixed + - (Issue #67) Saved Instagram posts not downloading + # 2022.8.28.0 *2022-08-28* diff --git a/ProgramScreenshots/FeedWindow.png b/ProgramScreenshots/FeedWindow.png new file mode 100644 index 0000000..dadde13 Binary files /dev/null and b/ProgramScreenshots/FeedWindow.png differ diff --git a/ProgramScreenshots/MainWindow.png b/ProgramScreenshots/MainWindow.png index 685229c..4bead73 100644 Binary files a/ProgramScreenshots/MainWindow.png and b/ProgramScreenshots/MainWindow.png differ diff --git a/ProgramScreenshots/MissingPosts.png b/ProgramScreenshots/MissingPosts.png new file mode 100644 index 0000000..e0904f1 Binary files /dev/null and b/ProgramScreenshots/MissingPosts.png differ diff --git a/ProgramScreenshots/SettingsAutoDownloader.png b/ProgramScreenshots/SettingsAutoDownloader.png index f95d35c..755f596 100644 Binary files a/ProgramScreenshots/SettingsAutoDownloader.png and b/ProgramScreenshots/SettingsAutoDownloader.png differ diff --git a/ProgramScreenshots/SettingsGlobalFeed.png b/ProgramScreenshots/SettingsGlobalFeed.png new file mode 100644 index 0000000..4565ff9 Binary files /dev/null and b/ProgramScreenshots/SettingsGlobalFeed.png differ diff --git a/ProgramsComparison.md b/ProgramsComparison.md index 3acca59..4cf7c4c 100644 --- a/ProgramsComparison.md +++ b/ProgramsComparison.md @@ -18,7 +18,7 @@ https://www.4kdownload.com/products/product-stogram | Download posts by location | No | **Yes** | | Save Private Instagram Content with Permission| Yes | Yes | | Download Instagram Stories and Highlights | Yes | Yes | -| See Others Instagram Feed As Your Own | No | **Yes** | +| See Others Instagram Feed As Your Own | Yes | Yes | | Download Instagram Video Posts | Yes | Yes | | Backup Your Instagram Account | Yes | Yes | | Save Instagram Posts by Date | Yes | Yes | @@ -31,7 +31,7 @@ https://www.4kdownload.com/products/product-stogram | Automatic Subscriptions Update | **Free** | Paid (43.56 EUR) | | Posts and Captions Export | No | Paid (43.56 EUR) | | Advertisements free | **No ADs at all for free** | Paid (14.52) | -| Operating Systems | Windows 7+ | Windows 7+, MacOS 10.13+, Ubuntu x64 | +| Operating Systems | Windows 10+ | Windows 7+, MacOS 10.13+, Ubuntu x64 | | Select want content type to download | **Yes** | No | | Instagram support | Yes | Yes | | Twitter support | **Yes** | No | @@ -66,9 +66,9 @@ https://github.com/RipMeApp/ripme | Export and import subscriptions | No | No | | **Paid** | **No** | **No** | | **Free options** | The program is completely free | The program is completely free, but site limits are not declared | -| Operating Systems | Windows 7+ | Windows, MacOS, Linux | +| Operating Systems | Windows 10+ | Windows, MacOS, Linux | | Select want content type to download | Yes | Yes | -| Suported sites | 3 internal and any site using plugins | 86+ sites (declared) | +| Suported sites | 6 internal and any site using plugins | 86+ sites (declared) | | Other sites support | **Yes** | No | | Still supported | **Yes** | **No (last release date May 4, 2021)** | diff --git a/README.md b/README.md index 0ecb298..6574433 100644 --- a/README.md +++ b/README.md @@ -35,6 +35,7 @@ Do you like this program? Consider adding to my coffee fund by making a donation - Add users from parsed channel - **Advanced user management** - **Automation** (downloading data automatically every ```X``` minutes) +- **Feed** (feed of downloaded media files) - Labeling users - Create download groups - Adding users to favorites and temporary diff --git a/SCrawler.Plugin.LPSG/My Project/AssemblyInfo.vb b/SCrawler.Plugin.LPSG/My Project/AssemblyInfo.vb index 6aa2f22..be84dec 100644 --- a/SCrawler.Plugin.LPSG/My Project/AssemblyInfo.vb +++ b/SCrawler.Plugin.LPSG/My Project/AssemblyInfo.vb @@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices ' by using the '*' as shown below: ' - - + + diff --git a/SCrawler.Plugin.LPSG/SiteSettings.vb b/SCrawler.Plugin.LPSG/SiteSettings.vb index 308f5db..080593b 100644 --- a/SCrawler.Plugin.LPSG/SiteSettings.vb +++ b/SCrawler.Plugin.LPSG/SiteSettings.vb @@ -97,4 +97,7 @@ Public Class SiteSettings : Implements ISiteSettings Public Function ReadyToDownload(ByVal What As ISiteSettings.Download) As Boolean Implements ISiteSettings.ReadyToDownload Return True End Function + Public Function GetUserPostUrl(ByVal UserID As String, ByVal PostID As String) As String Implements ISiteSettings.GetUserPostUrl + Return String.Empty + End Function End Class \ No newline at end of file diff --git a/SCrawler.Plugin.XVIDEOS/My Project/AssemblyInfo.vb b/SCrawler.Plugin.XVIDEOS/My Project/AssemblyInfo.vb index 5757cfc..67157ad 100644 --- a/SCrawler.Plugin.XVIDEOS/My Project/AssemblyInfo.vb +++ b/SCrawler.Plugin.XVIDEOS/My Project/AssemblyInfo.vb @@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices ' by using the '*' as shown below: ' - - + + diff --git a/SCrawler.Plugin.XVIDEOS/SiteSettings.vb b/SCrawler.Plugin.XVIDEOS/SiteSettings.vb index 37e6b87..404d6b0 100644 --- a/SCrawler.Plugin.XVIDEOS/SiteSettings.vb +++ b/SCrawler.Plugin.XVIDEOS/SiteSettings.vb @@ -179,4 +179,7 @@ Public Class SiteSettings : Implements ISiteSettings End If Return Nothing End Function + Public Function GetUserPostUrl(ByVal UserID As String, ByVal PostID As String) As String Implements ISiteSettings.GetUserPostUrl + Return String.Empty + End Function End Class \ No newline at end of file diff --git a/SCrawler.PluginProvider/Interfaces/ISiteSettings.vb b/SCrawler.PluginProvider/Interfaces/ISiteSettings.vb index eb42605..f214bcf 100644 --- a/SCrawler.PluginProvider/Interfaces/ISiteSettings.vb +++ b/SCrawler.PluginProvider/Interfaces/ISiteSettings.vb @@ -23,6 +23,7 @@ Namespace Plugin Function IsMyImageVideo(ByVal URL As String) As ExchangeOptions Function GetSpecialData(ByVal URL As String, ByVal Path As String, ByVal AskForPath As Boolean) As IEnumerable(Of PluginUserMedia) Function GetInstance(ByVal What As Download) As IPluginContentProvider + Function GetUserPostUrl(ByVal UserID As String, ByVal PostID As String) As String #Region "XML Support" Sub Load(ByVal XMLValues As IEnumerable(Of KeyValuePair(Of String, String))) #End Region diff --git a/SCrawler.PluginProvider/My Project/AssemblyInfo.vb b/SCrawler.PluginProvider/My Project/AssemblyInfo.vb index da50e0b..82d2157 100644 --- a/SCrawler.PluginProvider/My Project/AssemblyInfo.vb +++ b/SCrawler.PluginProvider/My Project/AssemblyInfo.vb @@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices ' by using the '*' as shown below: ' - - + + diff --git a/SCrawler.PluginProvider/Objects/PluginUserMedia.vb b/SCrawler.PluginProvider/Objects/PluginUserMedia.vb index 1180bdc..660c9d7 100644 --- a/SCrawler.PluginProvider/Objects/PluginUserMedia.vb +++ b/SCrawler.PluginProvider/Objects/PluginUserMedia.vb @@ -17,7 +17,7 @@ Namespace Plugin GIF = 50 m3u8 = 100 End Enum - Enum States As Integer : Unknown = 0 : Tried = 1 : Downloaded = 2 : Skipped = 3 : End Enum + Enum States As Integer : Unknown = 0 : Tried = 1 : Downloaded = 2 : Skipped = 3 : Missing = 4 : End Enum Public ContentType As Integer Public URL As String Public MD5 As String diff --git a/SCrawler/API/Base/SiteSettingsBase.vb b/SCrawler/API/Base/SiteSettingsBase.vb index db9c73b..329a6d8 100644 --- a/SCrawler/API/Base/SiteSettingsBase.vb +++ b/SCrawler/API/Base/SiteSettingsBase.vb @@ -11,12 +11,18 @@ Imports PersonalUtilities.Tools.WEB Imports SCrawler.Plugin Imports Download = SCrawler.Plugin.ISiteSettings.Download Namespace API.Base - Friend MustInherit Class SiteSettingsBase : Implements ISiteSettings + Friend MustInherit Class SiteSettingsBase : Implements ISiteSettings, IResponserContainer Friend ReadOnly Property Site As String Implements ISiteSettings.Site Friend Overridable ReadOnly Property Icon As Icon Implements ISiteSettings.Icon Friend Overridable ReadOnly Property Image As Image Implements ISiteSettings.Image Private Property Logger As ILogProvider = LogConnector Implements ISiteSettings.Logger Friend Overridable ReadOnly Property Responser As Response + Private Property IResponserContainer_Responser As Response Implements IResponserContainer.Responser + Get + Return Responser + End Get + Set : End Set + End Property Friend MustOverride Function GetInstance(ByVal What As Download) As IPluginContentProvider Implements ISiteSettings.GetInstance Friend Sub New(ByVal SiteName As String) Site = SiteName @@ -25,7 +31,15 @@ Namespace API.Base Site = SiteName Responser = New Response($"{SettingsFolderName}\Responser_{Site}.xml") With Responser - If .File.Exists Then .LoadSettings() Else .CookiesDomain = CookiesDomain : .Cookies = New CookieKeeper(.CookiesDomain) : .SaveSettings() + If .File.Exists Then + If EncryptCookies.CookiesEncrypted Then .CookiesEncryptKey = SettingsCLS.CookieEncryptKey + .LoadSettings() + Else + .CookiesDomain = CookiesDomain + .Cookies = New CookieKeeper(.CookiesDomain) With {.EncryptKey = SettingsCLS.CookieEncryptKey} + .CookiesEncryptKey = SettingsCLS.CookieEncryptKey + .SaveSettings() + End If End With End Sub #Region "XML" @@ -36,6 +50,7 @@ Namespace API.Base Friend Overridable Sub BeginInit() Implements ISiteSettings.BeginInit End Sub Friend Overridable Sub EndInit() Implements ISiteSettings.EndInit + EncryptCookies.ValidateCookiesEncrypt(Responser) End Sub Friend Overridable Sub BeginUpdate() Implements ISiteSettings.BeginUpdate End Sub @@ -63,6 +78,9 @@ Namespace API.Base End If Return String.Empty End Function + Friend Overridable Function GetUserPostUrl(ByVal UserID As String, ByVal PostID As String) As String Implements ISiteSettings.GetUserPostUrl + Return String.Empty + End Function Protected UserRegex As RParams = Nothing Friend Overridable Function IsMyUser(ByVal UserURL As String) As ExchangeOptions Implements ISiteSettings.IsMyUser Try diff --git a/SCrawler/API/Base/Structures.vb b/SCrawler/API/Base/Structures.vb index 3c425ae..39998f8 100644 --- a/SCrawler/API/Base/Structures.vb +++ b/SCrawler/API/Base/Structures.vb @@ -18,7 +18,7 @@ Namespace API.Base GIF = 50 m3u8 = 100 End Enum - Friend Enum States As Integer : Unknown = 0 : Tried = 1 : Downloaded = 2 : Skipped = 3 : End Enum + Friend Enum States As Integer : Unknown = 0 : Tried = 1 : Downloaded = 2 : Skipped = 3 : Missing = 4 : End Enum Friend [Type] As Types Friend URL_BASE As String Friend URL As String @@ -27,6 +27,7 @@ Namespace API.Base Friend Post As UserPost Friend PictureOption As String Friend State As States + Friend Attempts As Integer ''' ''' SomeFolder
''' SomeFolder\SomeFolder2 diff --git a/SCrawler/API/Base/UserDataBase.vb b/SCrawler/API/Base/UserDataBase.vb index be57d2a..85691c6 100644 --- a/SCrawler/API/Base/UserDataBase.vb +++ b/SCrawler/API/Base/UserDataBase.vb @@ -127,6 +127,8 @@ Namespace API.Base Private Const Name_DataMerging As String = "DataMerging" #Region "Downloaded data" Private Const Name_MediaType As String = "Type" + Private Const Name_MediaState As String = "State" + Private Const Name_MediaAttempts As String = "Attempts" Private Const Name_MediaURL As String = "URL" Private Const Name_MediaHash As String = "Hash" Private Const Name_MediaFile As String = "File" @@ -358,10 +360,31 @@ BlockNullPicture: Friend Overridable Property ReadyForDownload As Boolean = True Implements IUserData.ReadyForDownload Friend Property DownloadImages As Boolean = True Implements IUserData.DownloadImages Friend Property DownloadVideos As Boolean = True Implements IUserData.DownloadVideos + Friend Property DownloadMissingOnly As Boolean = False Implements IUserData.DownloadMissingOnly #End Region #Region "Content" Protected ReadOnly _ContentList As List(Of UserMedia) Protected ReadOnly _ContentNew As List(Of UserMedia) + Friend ReadOnly Property LatestData As List(Of UserMedia) + Protected ReadOnly MissingFinder As Predicate(Of UserMedia) = Function(c) c.State = UStates.Missing + Friend ReadOnly Property ContentMissing As List(Of UserMedia) + Get + If _ContentList.Count > 0 Then + Return _ContentList.Where(Function(c) MissingFinder(c)).ListIfNothing + Else + Return New List(Of UserMedia) + End If + End Get + End Property + Friend Overridable ReadOnly Property ContentMissingExists As Boolean + Get + Return _ContentList.Exists(MissingFinder) + End Get + End Property + Friend Sub RemoveMedia(ByVal m As UserMedia, ByVal State As UStates?) + Dim i% = If(State.HasValue, _ContentList.FindIndex(Function(mm) mm.State = State.Value And mm.Equals(m)), _ContentList.IndexOf(m)) + If i >= 0 Then _ContentList.RemoveAt(i) + End Sub Protected ReadOnly _TempMediaList As List(Of UserMedia) Protected ReadOnly _TempPostsList As List(Of String) Friend Function GetLastImageAddress() As SFile @@ -568,11 +591,12 @@ BlockNullPicture: Friend Sub New(Optional ByVal InvokeImageHandler As Boolean = True) _ContentList = New List(Of UserMedia) _ContentNew = New List(Of UserMedia) + LatestData = New List(Of UserMedia) _TempMediaList = New List(Of UserMedia) _TempPostsList = New List(Of String) Labels = New List(Of String) UserUpdatedEventHandlers = New List(Of IUserData.UserUpdatedEventHandler) - If InvokeImageHandler Then ImageHandler(Me) + If InvokeImageHandler Then MainFrameObj.ImageHandler(Me) End Sub Friend Sub SetEnvironment(ByRef h As SettingsHost, ByVal u As UserInfo, ByVal _LoadUserInformation As Boolean, Optional ByVal AttachUserInfo As Boolean = True) Implements IUserData.SetEnvironment @@ -583,13 +607,29 @@ BlockNullPicture: End If End Sub ''' - Friend Overloads Shared Function GetInstance(ByVal u As UserInfo, Optional ByVal _LoadUserInformation As Boolean = True) As IUserData + Friend Shared Function GetInstance(ByVal u As UserInfo, Optional ByVal _LoadUserInformation As Boolean = True) As IUserData If Not u.Plugin.IsEmptyString Then Return Settings(u.Plugin).GetInstance(u.DownloadOption, u, _LoadUserInformation) Else Throw New ArgumentOutOfRangeException("Plugin", $"Plugin [{u.Plugin}] information does not recognized by loader") End If End Function + Friend Shared Function GetPostUrl(ByVal u As IUserData, ByVal PostData As UserMedia) As String + Dim uName$ = String.Empty + Try + If Not u Is Nothing AndAlso Not u.IsCollection Then + With DirectCast(u, UserDataBase) + If Not .User.Plugin.IsEmptyString Then + uName = .User.Name + Return Settings(.User.Plugin).GetUserPostUrl(.ID, PostData.Post.ID) + End If + End With + End If + Return String.Empty + Catch ex As Exception + Return ErrorsDescriber.Execute(EDP.SendInLog, ex, $"GetPostUrl({uName}, {PostData.Post.ID})", String.Empty) + End Try + End Function #End Region #Region "Information & Content data files loader and saver" #Region "User information" @@ -676,19 +716,22 @@ BlockNullPicture: Protected MustOverride Sub LoadUserInformation_OptionalFields(ByRef Container As XmlFile, ByVal Loading As Boolean) #End Region #Region "User data" - Friend Overridable Overloads Sub LoadContentInformation() + Friend Overridable Overloads Sub LoadContentInformation(Optional ByVal Force As Boolean = False) Try UpdateDataFiles() - If Not MyFileData.Exists Then Exit Sub + If Not MyFileData.Exists Or (_DataLoaded And Not Force) Then Exit Sub Using x As New XmlFile(MyFileData, Protector.Modes.All, False) With {.XmlReadOnly = True, .AllowSameNames = True} x.LoadData() If x.Count > 0 Then + Dim fs$ = MyFile.CutPath.PathWithSeparator Dim gfn As Func(Of String, String) = Function(Input) If(Input.IsEmptyString, String.Empty, If(Input.Contains("\"), Input.CSFile.File, Input)) For Each v As EContainer In x _ContentList.Add(New UserMedia With { - .Type = AConvert(Of Integer)(v.Attribute(Name_MediaType).Value, 0), + .Type = v.Attribute(Name_MediaType).Value.FromXML(Of Integer)(CInt(UTypes.Undefined)), + .State = v.Attribute(Name_MediaState).Value.FromXML(Of Integer)(CInt(UStates.Downloaded)), + .Attempts = v.Attribute(Name_MediaAttempts).Value.FromXML(Of Integer)(0), .URL = v.Attribute(Name_MediaURL).Value, .URL_BASE = v.Value, .MD5 = v.Attribute(Name_MediaHash).Value, @@ -715,6 +758,8 @@ BlockNullPicture: If _ContentList.Count > 0 Then For Each i As UserMedia In _ContentList x.Add(New EContainer("MediaData", i.URL_BASE, {New EAttribute(Name_MediaType, CInt(i.Type)), + New EAttribute(Name_MediaState, CInt(i.State)), + New EAttribute(Name_MediaAttempts, i.Attempts), New EAttribute(Name_MediaURL, i.URL), New EAttribute(Name_MediaHash, i.MD5), New EAttribute(Name_MediaFile, i.File.File), @@ -764,6 +809,7 @@ BlockNullPicture: Dim UpPic As Boolean = Settings.ViewModeIsPicture AndAlso GetPicture(Of Image)(False) Is Nothing Dim sEnvir() As Boolean = {UserExists, UserSuspended} Dim EnvirChanged As Func(Of Boolean) = Function() Not sEnvir(0) = UserExists Or Not sEnvir(1) = UserSuspended + Dim _downContent As Func(Of UserMedia, Boolean) = Function(c) c.State = UStates.Downloaded UserExists = True UserSuspended = False DownloadedPictures(False) = 0 @@ -772,19 +818,25 @@ BlockNullPicture: _TempPostsList.Clear() Dim __SaveData As Boolean = Not CreatedByChannel Or Not Settings.FromChannelDownloadTopUse - If Not _DataLoaded Then LoadContentInformation() + LoadContentInformation() If MyFilePosts.Exists Then _TempPostsList.ListAddList(File.ReadAllLines(MyFilePosts)) If _ContentList.Count > 0 Then _TempPostsList.ListAddList(_ContentList.Select(Function(u) u.Post.ID), LNC) - ThrowAny(Token) - DownloadDataF(Token) - ThrowAny(Token) + If Not DownloadMissingOnly Then + ThrowAny(Token) + DownloadDataF(Token) + ThrowAny(Token) + Else + 'ReparseMissing(Token) + End If + _TempMediaList.ListAddList(ContentMissing, LNC) If _TempMediaList.Count > 0 Then If Not DownloadImages Then _TempMediaList.RemoveAll(Function(m) m.Type = UTypes.GIF Or m.Type = UTypes.Picture) If Not DownloadVideos Then _TempMediaList.RemoveAll(Function(m) m.Type = UTypes.Video Or m.Type = UTypes.VideoPre Or m.Type = UTypes.m3u8) + If DownloadMissingOnly Then _TempMediaList.RemoveAll(Function(m) Not m.State = UStates.Missing) End If ReparseVideo(Token) @@ -793,8 +845,9 @@ BlockNullPicture: _ContentNew.ListAddList(_TempMediaList, LAP.ClearBeforeAdd) DownloadContent(Token) ThrowIfDisposed() - _ContentList.ListAddList(_ContentNew.Where(Function(c) c.State = UStates.Downloaded), LNC) - If DownloadedTotal(False) > 0 Or EnvirChanged.Invoke Then + LatestData.ListAddList(_ContentNew.Where(_downContent), LNC) + _ContentList.ListAddList(_ContentNew.Where(_downContent), LNC) + If DownloadedTotal(False) > 0 Or EnvirChanged.Invoke Or _ContentList.Exists(MissingFinder) Then If __SaveData Then LastUpdated = Now RunScript() @@ -828,6 +881,7 @@ BlockNullPicture: _ContentNew.Clear() DownloadTopCount = Nothing DownloadToDate = Nothing + DownloadMissingOnly = False End Try End Sub Protected Function CheckDatesLimit(ByVal DateString As String, ByVal DateProvider As IFormatProvider) As Boolean @@ -853,7 +907,10 @@ BlockNullPicture: End If End Sub Protected MustOverride Sub DownloadDataF(ByVal Token As CancellationToken) - Protected MustOverride Sub ReparseVideo(ByVal Token As CancellationToken) + Protected Overridable Sub ReparseVideo(ByVal Token As CancellationToken) + End Sub + Protected Overridable Sub ReparseMissing(ByVal Token As CancellationToken) + End Sub Protected MustOverride Sub DownloadContent(ByVal Token As CancellationToken) Protected Sub DownloadContentDefault(ByVal Token As CancellationToken) Try @@ -864,6 +921,7 @@ BlockNullPicture: _ContentNew.RemoveAll(Function(c) c.URL.IsEmptyString) If _ContentNew.Count > 0 Then MyFile.Exists(SFO.Path) + Dim MissingErrorsAdd As Boolean = Settings.AddMissingErrorsToLog Dim MyDir$ = MyFile.CutPath.PathNoSeparator Dim vsf As Boolean = SeparateVideoFolderF Dim __isVideo As Boolean @@ -871,7 +929,7 @@ BlockNullPicture: Dim v As UserMedia Using w As New WebClient If vsf Then SFileShares.SFileExists($"{MyDir}\Video\", SFO.Path) - Progress.TotalCount += _ContentNew.Count + Progress.Maximum += _ContentNew.Count For i = 0 To _ContentNew.Count - 1 ThrowAny(Token) v = _ContentNew(i) @@ -922,7 +980,9 @@ BlockNullPicture: v.State = UStates.Downloaded dCount += 1 Catch wex As Exception - ErrorDownloading(f, v.URL_BASE) + v.Attempts += 1 + v.State = UStates.Missing + If MissingErrorsAdd Then ErrorDownloading(f, v.URL_BASE) End Try Else v.State = UStates.Skipped @@ -1000,7 +1060,7 @@ BlockNullPicture: Friend Overridable Function Delete() As Integer Implements IUserData.Delete Dim f As SFile = SFile.GetPath(MyFile.CutPath.Path) If f.Exists(SFO.Path, False) AndAlso (User.Merged OrElse f.Delete(SFO.Path, Settings.DeleteMode)) Then - ImageHandler(Me, False) + MainFrameObj.ImageHandler(Me, False) Settings.UsersList.Remove(User) Settings.UpdateUsersList() Settings.Users.Remove(Me) @@ -1205,6 +1265,7 @@ BlockNullPicture: If disposing Then _ContentList.Clear() _ContentNew.Clear() + LatestData.Clear() _TempMediaList.Clear() _TempPostsList.Clear() If Not Responser Is Nothing Then Responser.Dispose() @@ -1268,6 +1329,7 @@ BlockNullPicture: ReadOnly Property Key As String Property DownloadImages As Boolean Property DownloadVideos As Boolean + Property DownloadMissingOnly As Boolean Property ScriptUse As Boolean Property ScriptData As String Function GetLVI(ByVal Destination As ListView) As ListViewItem diff --git a/SCrawler/API/Instagram/SiteSettings.vb b/SCrawler/API/Instagram/SiteSettings.vb index 27249f8..061de53 100644 --- a/SCrawler/API/Instagram/SiteSettings.vb +++ b/SCrawler/API/Instagram/SiteSettings.vb @@ -186,30 +186,23 @@ Namespace API.Instagram End With End Sub #End Region - Friend Overrides ReadOnly Property Responser As Response Private Initialized As Boolean = False #End Region #Region "Initializer" Friend Sub New(ByRef _XML As XmlFile, ByVal GlobalPath As SFile) - MyBase.New(InstagramSite) - Responser = New Response($"{SettingsFolderName}\Responser_{Site}.xml") + MyBase.New(InstagramSite, "instagram.com") Dim app_id$ = String.Empty Dim www_claim$ = String.Empty Dim token$ = String.Empty With Responser - If .File.Exists Then - .LoadSettings() + If .Headers.Count > 0 Then With .Headers If .ContainsKey(Header_CSRF_TOKEN) Then token = .Item(Header_CSRF_TOKEN) If .ContainsKey(Header_IG_APP_ID) Then app_id = .Item(Header_IG_APP_ID) If .ContainsKey(Header_IG_WWW_CLAIM) Then www_claim = .Item(Header_IG_WWW_CLAIM) End With - Else - .CookiesDomain = "instagram.com" - .Cookies = New CookieKeeper(.CookiesDomain) - .SaveSettings() End If End With @@ -250,6 +243,7 @@ Namespace API.Instagram End Sub Friend Overrides Sub EndInit() Initialized = True + MyBase.EndInit() End Sub #End Region #Region "PropertiesDataChecker" diff --git a/SCrawler/API/Instagram/UserData.vb b/SCrawler/API/Instagram/UserData.vb index 30f41cc..75889fc 100644 --- a/SCrawler/API/Instagram/UserData.vb +++ b/SCrawler/API/Instagram/UserData.vb @@ -255,7 +255,7 @@ Namespace API.Instagram 'Check environment If Cursor.IsEmptyString And _InstaHash.IsEmptyString Then _ - _InstaHash = CStr(If(IsSavedPosts, MySiteSettings.HashSavedPosts, MySiteSettings.Hash).Value) + _InstaHash = CStr(If(IsSavedPosts, MySiteSettings.HashSavedPosts, MySiteSettings.Hash).Value) If ID.IsEmptyString Then GetUserId() If ID.IsEmptyString Then Throw New ArgumentException("User ID is not detected", "ID") @@ -317,13 +317,16 @@ Namespace API.Instagram If IsSavedPosts Then PostID = node.Value("shortcode") If Not PostID.IsEmptyString Then - If _TempPostsList.Contains(PostID) Then Throw New ExitException(_DownloadComplete) Else _SavedPostsIDs.Add(PostID) + If _TempPostsList.Contains(PostID) Then Throw New ExitException(_DownloadComplete) 'Else _SavedPostsIDs.Add(PostID) End If + End If + PostID = node.Value("id") + If Not PostID.IsEmptyString And _TempPostsList.Contains(PostID) Then Throw New ExitException(_DownloadComplete) + _TempPostsList.Add(PostID) + PostDate = node.Value("taken_at_timestamp") + If IsSavedPosts Then + _SavedPostsIDs.Add(PostID) Else - PostID = node.Value("id") - If Not PostID.IsEmptyString And _TempPostsList.Contains(PostID) Then Throw New ExitException(_DownloadComplete) - _TempPostsList.Add(PostID) - PostDate = node.Value("taken_at_timestamp") If Not CheckDatesLimit(PostDate, DateProvider) Then Throw New ExitException(_DownloadComplete) ObtainMedia(node, PostID, PostDate, SpecFolder) End If @@ -394,7 +397,8 @@ Namespace API.Instagram Dim e As New ErrorsDescriber(EDP.ThrowException) For i% = _Index To _SavedPostsIDs.Count - 1 _Index = i - URL = $"https://instagram.com/p/{_SavedPostsIDs(i)}/?__a=1" + 'URL = $"https://instagram.com/p/{_SavedPostsIDs(i)}/?__a=1" + URL = $"https://i.instagram.com/api/v1/media/{_SavedPostsIDs(i)}/info/" ThrowAny(Token) NextRequest(((i + 1) Mod 5) = 0) ThrowAny(Token) @@ -437,8 +441,26 @@ Namespace API.Instagram ProcessException(DoEx, Token, $"downloading saved posts error [{URL}]") End Try End Sub - Protected Overrides Sub ReparseVideo(ByVal Token As CancellationToken) - End Sub +#End Region +#Region "Code ID converters" + Friend Shared Function CodeToID(ByVal Code As String) As String + Const CodeSymbols$ = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_" + Try + If Not Code.IsEmptyString Then + Dim c As Char + Dim id& = 0 + For i% = 0 To Code.Length - 1 + c = Code(i) + id = (id * 64) + CodeSymbols.IndexOf(c) + Next + Return id + Else + Return String.Empty + End If + Catch ex As Exception + Return ErrorsDescriber.Execute(EDP.SendInLog, ex, $"[API.Instagram.UserData.CodeToID({Code})", String.Empty) + End Try + End Function #End Region #Region "Obtain Media" Private Sub ObtainMedia(ByVal node As EContainer, ByVal PostID As String, ByVal PostDate As String, ByVal SpecFolder As String) @@ -690,6 +712,7 @@ Namespace API.Instagram Try If Not URL.IsEmptyString AndAlso URL.Contains("instagram.com") Then Dim PID$ = RegexReplace(URL, RParams.DMS(".*?instagram.com/p/([_\w\d]+)", 1)) + If Not PID.IsEmptyString AndAlso Not ACheck(Of Long)(PID) Then PID = CodeToID(PID) If Not PID.IsEmptyString Then Using t As New UserData t.SetEnvironment(Settings(_Settings.GetType.GetCustomAttribute(Of Plugin.Attributes.Manifest)().GUID), Nothing, False, False) diff --git a/SCrawler/API/Reddit/SiteSettings.vb b/SCrawler/API/Reddit/SiteSettings.vb index b07ad5e..6ef82f2 100644 --- a/SCrawler/API/Reddit/SiteSettings.vb +++ b/SCrawler/API/Reddit/SiteSettings.vb @@ -30,20 +30,11 @@ Namespace API.Reddit Friend ReadOnly Property SavedPostsUserName As PropertyValue Friend ReadOnly Property UseM3U8 As PropertyValue - Friend Overrides ReadOnly Property Responser As Response Friend Sub New() - MyBase.New(RedditSite) - Responser = New Response($"{SettingsFolderName}\Responser_{Site}.xml") - + MyBase.New(RedditSite, "reddit.com") With Responser - If .File.Exists Then - .LoadSettings() - Else - .CookiesDomain = "reddit.com" - .Cookies = New CookieKeeper(.CookiesDomain) - .Decoders.Add(SymbolsConverter.Converters.Unicode) - .SaveSettings() - End If + If .Decoders.Count = 0 OrElse Not .Decoders.Contains(SymbolsConverter.Converters.Unicode) Then _ + .Decoders.Add(SymbolsConverter.Converters.Unicode) : .SaveSettings() End With SavedPostsUserName = New PropertyValue(String.Empty, GetType(String)) UseM3U8 = New PropertyValue(True) @@ -110,5 +101,8 @@ Namespace API.Reddit Using f As New RedditViewSettingsForm(Options) : f.ShowDialog() : End Using End If End Sub + Friend Overrides Function GetUserPostUrl(ByVal UserID As String, ByVal PostID As String) As String + Return $"https://www.reddit.com/comments/{PostID.Split("_").LastOrDefault}/" + End Function End Class End Namespace \ No newline at end of file diff --git a/SCrawler/API/Reddit/UserData.vb b/SCrawler/API/Reddit/UserData.vb index f9632b7..d7908b2 100644 --- a/SCrawler/API/Reddit/UserData.vb +++ b/SCrawler/API/Reddit/UserData.vb @@ -150,6 +150,7 @@ Namespace API.Reddit End Sub Protected Overrides Sub DownloadDataF(ByVal Token As CancellationToken) _TotalPostsDownloaded = 0 + 'If Not IsSavedPosts AndAlso (Not IsChannel OrElse ChannelInfo Is Nothing) Then ReparseMissing(Token) If IsSavedPosts Then DownloadDataChannel(String.Empty, Token) ElseIf IsChannel Then @@ -507,6 +508,72 @@ Namespace API.Reddit ProcessException(ex, Token, "video reparsing error", False) End Try End Sub + Protected Overrides Sub ReparseMissing(ByVal Token As CancellationToken) + Dim rList As New List(Of Integer) + Try + If _ContentList.Exists(MissingFinder) Then + Dim m As UserMedia + Dim j As EContainer, ss As EContainer + Dim r$, tmpUrl$, PostDate$, _UserID$ + Dim err As New ErrorsDescriber(EDP.ReturnValue) + Dim node As Object() = {"data", "children", 0, "data"} + Dim eCount As Predicate(Of EContainer) = Function(e) e.Count > 0 + Dim cItems As Predicate(Of EContainer) = Function(e) If(e.ItemF(node)?.Count, 0) > 0 + For i% = 0 To _ContentList.Count - 1 + m = _ContentList(i) + If m.State = UStates.Missing AndAlso Not m.Post.ID.IsEmptyString Then + ThrowAny(Token) + r = Responser.GetResponse($"https://www.reddit.com/comments/{m.Post.ID.Split("_").LastOrDefault}/.json",, err) + If Not r.IsEmptyString Then + j = JsonDocument.Parse(r, err) + If Not j Is Nothing Then + If j.Contains(cItems) Then + With j.ItemF({cItems}).ItemF(node) + If .Contains("created") Then PostDate = .Item("created").Value Else PostDate = String.Empty + _UserID = .Value("author") + tmpUrl = .Value("url") + If Not tmpUrl.IsEmptyString AndAlso tmpUrl.StringContains({"redgifs.com", "gfycat.com"}) Then + _TempMediaList.ListAddValue(MediaFromData(UTypes.VideoPre, tmpUrl, m.Post.ID, PostDate, _UserID, IsChannel), LNC) + _TotalPostsDownloaded += 1 + ElseIf Not .Value({"media", "reddit_video"}, "fallback_url").IsEmptyString Then + tmpUrl = .Value({"media", "reddit_video"}, "fallback_url") + If UseM3U8 AndAlso Not .Value({"media", "reddit_video"}, "hls_url").IsEmptyString Then + _TempMediaList.ListAddValue(MediaFromData(UTypes.m3u8, .Value({"media", "reddit_video"}, "hls_url"), + m.Post.ID, PostDate, _UserID, IsChannel), LNC) + Else + '_TempMediaList.ListAddValue(MediaFromData(UTypes.VideoPre + UTypes.m3u8, tmpUrl, PostID, PostDate, _UserID, IsChannel), LNC) + _TempMediaList.ListAddValue(MediaFromData(UTypes.Video, tmpUrl, m.Post.ID, PostDate, _UserID, IsChannel), LNC) + _TotalPostsDownloaded += 1 + End If + ElseIf CreateImgurMedia(tmpUrl, m.Post.ID, PostDate, _UserID, IsChannel) Then + _TotalPostsDownloaded += 1 + ElseIf If(.Item("media_metadata")?.Count, 0) > 0 Then + DownloadGallery(.Self, m.Post.ID, PostDate, _UserID, SaveToCache) + _TotalPostsDownloaded += 1 + ElseIf .Contains("preview") Then + ss = .ItemF({"preview", "images", eCount, "source", "url"}).XmlIfNothing + If Not ss.Value.IsEmptyString Then + _TempMediaList.ListAddValue(MediaFromData(UTypes.Picture, ss.Value, m.Post.ID, PostDate, _UserID, IsChannel), LNC) + _TotalPostsDownloaded += 1 + End If + End If + End With + End If + j.Dispose() + End If + End If + End If + Next + End If + Catch ex As Exception + ProcessException(ex, Token, "missing data downloading error") + Finally + If rList.Count > 0 Then + For i% = rList.Count - 1 To 0 Step -1 : _ContentList.RemoveAt(rList(i)) : Next + rList.Clear() + End If + End Try + End Sub Friend Shared Function GetVideoInfo(ByVal URL As String, ByVal resp As Response) As IEnumerable(Of UserMedia) Try If Not URL.IsEmptyString AndAlso URL.Contains("redgifs") Then @@ -561,6 +628,8 @@ Namespace API.Reddit _ContentNew.RemoveAll(Function(c) c.URL.IsEmptyString) If _ContentNew.Count > 0 Then MyFile.Exists(SFO.Path) + Dim MissingErrorsAdd As Boolean = Settings.AddMissingErrorsToLog + Dim IsImgurStuff As Boolean Dim MyDir$ If Not IsSavedPosts AndAlso (IsChannel And SaveToCache And Not ChannelInfo Is Nothing) Then MyDir = ChannelInfo.CachePath.PathNoSeparator @@ -629,7 +698,7 @@ Namespace API.Reddit Dim m$ Using w As New WebClient If vsf Then SFileShares.SFileExists($"{MyDir}\Video\", SFO.Path) - Progress.TotalCount += _ContentNew.Count + Progress.Maximum += _ContentNew.Count For i = 0 To _ContentNew.Count - 1 ThrowAny(Token) v = _ContentNew(i) @@ -651,6 +720,7 @@ Namespace API.Reddit If (Not m.IsEmptyString AndAlso Not HashList.Contains(m)) Or Not (v.Type = UTypes.Picture Or v.Type = UTypes.GIF) Or Not UseMD5 Or ImgurUrls.Count > 0 Then + isImgurStuff = ImgurUrls.Count > 0 Do If Not cached And Not m.IsEmptyString Then HashList.Add(m) v.MD5 = m @@ -695,7 +765,7 @@ Namespace API.Reddit dCount += 1 End If Catch wex As Exception - If Not IsChannel Then ErrorDownloading(f, v.URL) + If Not IsChannel And Not IsImgurStuff And MissingErrorsAdd Then ErrorDownloading(f, v.URL) End Try If ImgurUrls.Count > 0 Then ImgurUrls.RemoveAt(0) Loop While ImgurUrls.Count > 0 diff --git a/SCrawler/API/Redgifs/SiteSettings.vb b/SCrawler/API/Redgifs/SiteSettings.vb index 3fe11d2..339d9d8 100644 --- a/SCrawler/API/Redgifs/SiteSettings.vb +++ b/SCrawler/API/Redgifs/SiteSettings.vb @@ -15,12 +15,12 @@ Namespace API.RedGifs Friend Class SiteSettings : Inherits SiteSettingsBase Friend Overrides ReadOnly Property Icon As Icon Get - Return My.Resources.RegGifsIcon + Return My.Resources.RedGifsIcon End Get End Property Friend Overrides ReadOnly Property Image As Image Get - Return My.Resources.RegGifsPic32 + Return My.Resources.RedGifsPic32 End Get End Property Friend Sub New() @@ -35,5 +35,11 @@ Namespace API.RedGifs Friend Overrides Function GetSpecialDataF(ByVal URL As String) As IEnumerable(Of UserMedia) Return Reddit.UserData.GetVideoInfo(URL, Nothing) End Function + Friend Overrides Function GetUserPostUrl(ByVal UserID As String, ByVal PostID As String) As String + Return $"https://www.redgifs.com/watch/{PostID}" + End Function + Friend Overrides Function Available(ByVal What As ISiteSettings.Download, ByVal Silent As Boolean) As Boolean + Return False + End Function End Class End Namespace \ No newline at end of file diff --git a/SCrawler/API/Redgifs/UserData.vb b/SCrawler/API/Redgifs/UserData.vb index 9a59d14..03c26e8 100644 --- a/SCrawler/API/Redgifs/UserData.vb +++ b/SCrawler/API/Redgifs/UserData.vb @@ -13,6 +13,7 @@ Imports System.Net Imports System.Threading Imports SCrawler.API.Base Imports UTypes = SCrawler.API.Base.UserMedia.Types +Imports UStates = SCrawler.API.Base.UserMedia.States Namespace API.RedGifs Friend Class UserData : Inherits UserDataBase Friend Sub New() @@ -20,6 +21,7 @@ Namespace API.RedGifs Protected Overrides Sub LoadUserInformation_OptionalFields(ByRef Container As XmlFile, ByVal Loading As Boolean) End Sub Protected Overrides Sub DownloadDataF(ByVal Token As CancellationToken) + ReparseMissing(Token) DownloadData(1, Token) End Sub Private Overloads Sub DownloadData(ByVal Page As Integer, ByVal Token As CancellationToken) @@ -29,8 +31,6 @@ Namespace API.RedGifs Dim r$ = Responser.GetResponse(URL,, EDP.ThrowException) Dim postDate$, postID$ Dim pTotal% = 0 - Dim u$ - Dim ut As UTypes If Not r.IsEmptyString Then Using j As EContainer = JsonDocument.Parse(r).XmlIfNothing If j.Contains("gifs") Then @@ -40,21 +40,7 @@ Namespace API.RedGifs If Not CheckDatesLimit(postDate, DateProvider) Then Exit Sub postID = g.Value("id") If Not _TempPostsList.Contains(postID) Then _TempPostsList.Add(postID) Else Exit For - With g("urls") - If .ListExists Then - u = If(.Item("hd"), .Item("sd")).XmlIfNothingValue - If Not u.IsEmptyString Then - ut = UTypes.Undefined - 'Type 1: video - 'Type 2: image - Select Case g.Value("type").FromXML(Of Integer)(0) - Case 1 : ut = UTypes.Video - Case 2 : ut = UTypes.Picture - End Select - If Not ut = UTypes.Undefined Then _TempMediaList.ListAddValue(MediaFromData(ut, u, postID, postDate)) - End If - End If - End With + ObtainMedia(g, postID, postDate) Next End If End Using @@ -64,16 +50,84 @@ Namespace API.RedGifs ProcessException(ex, Token, $"data downloading error [{URL}]") End Try End Sub - Protected Overrides Sub ReparseVideo(ByVal Token As CancellationToken) + Private Sub ObtainMedia(ByVal j As EContainer, ByVal PostID As String, + Optional ByVal PostDateStr As String = Nothing, Optional ByVal PostDateDate As Date? = Nothing, + Optional ByVal State As UStates = UStates.Unknown) + With j("urls") + If .ListExists Then + Dim u$ = If(.Item("hd"), .Item("sd")).XmlIfNothingValue + If Not u.IsEmptyString Then + Dim ut As UTypes = UTypes.Undefined + 'Type 1: video + 'Type 2: image + Select Case j.Value("type").FromXML(Of Integer)(0) + Case 1 : ut = UTypes.Video + Case 2 : ut = UTypes.Picture + End Select + If Not ut = UTypes.Undefined Then _TempMediaList.ListAddValue(MediaFromData(ut, u, PostID, PostDateStr, PostDateDate, State)) + End If + End If + End With + End Sub + Protected Overrides Sub ReparseMissing(ByVal Token As CancellationToken) + Dim rList As New List(Of Integer) + Try + If _ContentList.Exists(MissingFinder) Then + Dim url$, r$ + Dim u As UserMedia + Dim j As EContainer + For i% = 0 To _ContentList.Count - 1 + If _ContentList(i).State = UserMedia.States.Missing Then + ThrowAny(Token) + u = _ContentList(i) + If Not u.Post.ID.IsEmptyString Then + url = $"https://api.redgifs.com/v2/gifs/{u.Post.ID}?views=yes&users=yes" + Try + r = Responser.GetResponse(url,, EDP.ThrowException) + If Not r.IsEmptyString Then + j = JsonDocument.Parse(r) + If Not j Is Nothing Then + If If(j("gif")?.Count, 0) > 0 Then + ObtainMedia(j("gif"), u.Post.ID,, u.Post.Date, UStates.Missing) + rList.Add(i) + End If + End If + End If + Catch down_ex As Exception + u.Attempts += 1 + _ContentList(i) = u + End Try + Else + rList.Add(i) + End If + End If + Next + End If + Catch dex As ObjectDisposedException When Disposed + Catch ex As Exception + ProcessException(ex, Token, $"missing data downloading error") + Finally + If Not Disposed And rList.Count > 0 Then + For i% = rList.Count - 1 To 0 Step -1 : _ContentList.RemoveAt(rList(i)) : Next + End If + End Try End Sub Protected Overrides Sub DownloadContent(ByVal Token As CancellationToken) DownloadContentDefault(Token) End Sub - Private Shared Function MediaFromData(ByVal t As UTypes, ByVal _URL As String, ByVal PostID As String, ByVal PostDate As String) As UserMedia + Private Shared Function MediaFromData(ByVal t As UTypes, ByVal _URL As String, ByVal PostID As String, + ByVal PostDateStr As String, ByVal PostDateDate As Date?, ByVal State As UStates) As UserMedia _URL = LinkFormatterSecure(RegexReplace(_URL.Replace("\", String.Empty), LinkPattern)) Dim m As New UserMedia(_URL, t) With {.Post = New UserPost With {.ID = PostID}} If Not m.URL.IsEmptyString Then m.File = CStr(RegexReplace(m.URL, FilesPattern)) : m.URL_BASE = m.URL - If Not PostDate.IsEmptyString Then m.Post.Date = AConvert(Of Date)(PostDate, DateProvider, Nothing) Else m.Post.Date = Nothing + If Not PostDateStr.IsEmptyString Then + m.Post.Date = AConvert(Of Date)(PostDateStr, DateProvider, Nothing) + ElseIf PostDateDate.HasValue Then + m.Post.Date = PostDateDate + Else + m.Post.Date = Nothing + End If + m.State = State Return m End Function Protected Overrides Function DownloadingException(ByVal ex As Exception, ByVal Message As String, Optional ByVal FromPE As Boolean = False) As Integer diff --git a/SCrawler/API/Twitter/SiteSettings.vb b/SCrawler/API/Twitter/SiteSettings.vb index 479171b..731944b 100644 --- a/SCrawler/API/Twitter/SiteSettings.vb +++ b/SCrawler/API/Twitter/SiteSettings.vb @@ -43,6 +43,7 @@ Namespace API.Twitter With Responser If .File.Exists Then + If EncryptCookies.CookiesEncrypted Then .CookiesEncryptKey = SettingsCLS.CookieEncryptKey .LoadSettings() With .Headers If .ContainsKey(Header_Authorization) Then a = .Item(Header_Authorization) @@ -52,7 +53,8 @@ Namespace API.Twitter .ContentType = "application/json" .Accept = "*/*" .CookiesDomain = "twitter.com" - .Cookies = New CookieKeeper(.CookiesDomain) + .Cookies = New CookieKeeper(.CookiesDomain) With {.EncryptKey = SettingsCLS.CookieEncryptKey} + .CookiesEncryptKey = SettingsCLS.CookieEncryptKey .Decoders.Add(SymbolsConverter.Converters.Unicode) With .Headers .Add("sec-ch-ua", " Not;A Brand"";v=""99"", ""Google Chrome"";v=""91"", ""Chromium"";v=""91""") @@ -101,5 +103,8 @@ Namespace API.Twitter Friend Overrides Function GetSpecialDataF(ByVal URL As String) As IEnumerable(Of UserMedia) Return UserData.GetVideoInfo(URL, Responser) End Function + Friend Overrides Function GetUserPostUrl(ByVal UserID As String, ByVal PostID As String) As String + Return $"https://twitter.com/{UserID}/status/{PostID}" + End Function End Class End Namespace \ No newline at end of file diff --git a/SCrawler/API/Twitter/UserData.vb b/SCrawler/API/Twitter/UserData.vb index 7f2a9f8..54ceda7 100644 --- a/SCrawler/API/Twitter/UserData.vb +++ b/SCrawler/API/Twitter/UserData.vb @@ -13,6 +13,7 @@ Imports PersonalUtilities.Functions.RegularExpressions Imports System.Net Imports System.Threading Imports SCrawler.API.Base +Imports UStates = SCrawler.API.Base.UserMedia.States Namespace API.Twitter Friend Class UserData : Inherits UserDataBase #Region "Declarations" @@ -33,6 +34,7 @@ Namespace API.Twitter Else If _ContentList.Count > 0 Then _DataNames.ListAddList(_ContentList.Select(Function(c) c.File.File), LAP.ClearBeforeAdd, LAP.NotContainsOnly) DownloadData(String.Empty, Token) + 'ReparseMissing(Token) End If End Sub Private Overloads Sub DownloadData(ByVal POST As String, ByVal Token As CancellationToken) @@ -41,8 +43,8 @@ Namespace API.Twitter Dim NextCursor$ = String.Empty Dim __NextCursor As Predicate(Of EContainer) = Function(e) e.Value({"content", "operation", "cursor"}, "cursorType") = "Bottom" Dim PostID$ = String.Empty - Dim PostDate$, dName$ - Dim m As EContainer, nn As EContainer, s As EContainer + Dim PostDate$ ', dName$ + Dim nn As EContainer, s As EContainer ', m As EContainer Dim NewPostDetected As Boolean = False Dim ExistsDetected As Boolean = False @@ -96,22 +98,23 @@ Namespace API.Twitter If IsSavedPosts OrElse Not ParseUserMediaOnly OrElse (Not nn.Contains("retweeted_status") OrElse (Not ID.IsEmptyString AndAlso UID(nn("retweeted_status")) = ID)) Then - If Not CheckVideoNode(nn, PostID, PostDate) Then - s = nn.ItemF({"extended_entities", "media"}) - If s Is Nothing OrElse s.Count = 0 Then s = nn.ItemF({"retweeted_status", "extended_entities", "media"}) - If Not s Is Nothing AndAlso s.Count > 0 Then - For Each m In s - If m.Count > 0 AndAlso m.Contains("media_url") Then - dName = UrlFile(m("media_url").Value) - If Not dName.IsEmptyString AndAlso Not _DataNames.Contains(dName) Then - _DataNames.Add(dName) - _TempMediaList.ListAddValue(MediaFromData(m("media_url").Value, - PostID, PostDate, GetPictureOption(m)), LNC) - End If - End If - Next - End If - End If + 'If Not CheckVideoNode(nn, PostID, PostDate) Then + ' s = nn.ItemF({"extended_entities", "media"}) + ' If s Is Nothing OrElse s.Count = 0 Then s = nn.ItemF({"retweeted_status", "extended_entities", "media"}) + ' If Not s Is Nothing AndAlso s.Count > 0 Then + ' For Each m In s + ' If m.Count > 0 AndAlso m.Contains("media_url") Then + ' dName = UrlFile(m("media_url").Value) + ' If Not dName.IsEmptyString AndAlso Not _DataNames.Contains(dName) Then + ' _DataNames.Add(dName) + ' _TempMediaList.ListAddValue(MediaFromData(m("media_url").Value, + ' PostID, PostDate, GetPictureOption(m)), LNC) + ' End If + ' End If + ' Next + ' End If + 'End If + ObtainMedia(nn, PostID, PostDate) End If End If Next @@ -136,6 +139,61 @@ Namespace API.Twitter ProcessException(ex, Token, $"data downloading error{IIf(IsSavedPosts, " (Saved Posts)", String.Empty)} [{URL}]") End Try End Sub + Private Sub ObtainMedia(ByVal e As EContainer, ByVal PostID As String, ByVal PostDate As String, Optional ByVal State As UStates = UStates.Unknown) + If Not CheckVideoNode(e, PostID, PostDate) Then + Dim s As EContainer = e.ItemF({"extended_entities", "media"}) + If s Is Nothing OrElse s.Count = 0 Then s = e.ItemF({"retweeted_status", "extended_entities", "media"}) + If If(s?.Count, 0) > 0 Then + For Each m In s + If m.Contains("media_url") Then + Dim dName$ = UrlFile(m("media_url").Value) + If Not dName.IsEmptyString AndAlso Not _DataNames.Contains(dName) Then + _DataNames.Add(dName) + _TempMediaList.ListAddValue(MediaFromData(m("media_url").Value, + PostID, PostDate, GetPictureOption(m), State), LNC) + End If + End If + Next + End If + End If + End Sub + Protected Overrides Sub ReparseMissing(ByVal Token As CancellationToken) + Dim rList As New List(Of Integer) + Dim URL$ = String.Empty + Try + If ContentMissingExists Then + Dim m As UserMedia + Dim r$, PostDate$ + Dim j As EContainer + For i% = 0 To _ContentList.Count - 1 + If _ContentList(i).State = UStates.Missing Then + m = _ContentList(i) + If Not m.Post.ID.IsEmptyString Then + ThrowAny(Token) + URL = $"https://api.twitter.com/1.1/statuses/show.json?id={m.Post.ID}" + r = Responser.GetResponse(URL,, EDP.ReturnValue) + If Not r.IsEmptyString Then + j = JsonDocument.Parse(r) + If Not j Is Nothing Then + PostDate = String.Empty + If j.Contains("created_at") Then PostDate = j("created_at").Value Else PostDate = String.Empty + ObtainMedia(j, m.Post.ID, PostDate, UStates.Missing) + rList.Add(i) + End If + End If + End If + End If + Next + End If + Catch ex As Exception + ProcessException(ex, Token, $"ReparseMissing error [{URL}]") + Finally + If rList.Count > 0 Then + For i% = rList.Count - 1 To 0 Step -1 : _ContentList.RemoveAt(i) : Next + rList.Clear() + End If + End Try + End Sub Friend Shared Function GetVideoInfo(ByVal URL As String, ByVal resp As Response) As IEnumerable(Of UserMedia) Try If URL.Contains("twitter") Then @@ -268,8 +326,6 @@ Namespace API.Twitter End If Return String.Empty End Function - Protected Overrides Sub ReparseVideo(ByVal Token As CancellationToken) - End Sub Private Function UrlFile(ByVal URL As String) As String Try Dim f As SFile = CStr(RegexReplace(LinkFormatterSecure(RegexReplace(URL.Replace("\", String.Empty), LinkPattern)), FilesPattern)) @@ -280,7 +336,8 @@ Namespace API.Twitter End Function #End Region Private Shared Function MediaFromData(ByVal _URL As String, ByVal PostID As String, ByVal PostDate As String, - Optional ByVal _PictureOption As String = "") As UserMedia + Optional ByVal _PictureOption As String = Nothing, + Optional ByVal State As UStates = UStates.Unknown) As UserMedia _URL = LinkFormatterSecure(RegexReplace(_URL.Replace("\", String.Empty), LinkPattern)) Dim m As New UserMedia(_URL) With {.PictureOption = _PictureOption, .Post = New UserPost With {.ID = PostID}} If Not m.URL.IsEmptyString Then m.File = CStr(RegexReplace(m.URL, FilesPattern)) @@ -288,6 +345,7 @@ Namespace API.Twitter m.URL_BASE = $"{m.URL.Replace($".{m.File.Extension}", String.Empty)}?format={m.File.Extension}&name={m.PictureOption}" End If If Not PostDate.IsEmptyString Then m.Post.Date = AConvert(Of Date)(PostDate, Declarations.DateProvider, Nothing) Else m.Post.Date = Nothing + m.State = State Return m End Function #End Region diff --git a/SCrawler/API/UserDataBind.vb b/SCrawler/API/UserDataBind.vb index bd717f5..72955e8 100644 --- a/SCrawler/API/UserDataBind.vb +++ b/SCrawler/API/UserDataBind.vb @@ -92,6 +92,11 @@ Namespace API End If End Get End Property + Friend Overrides ReadOnly Property ContentMissingExists As Boolean + Get + Return Count > 0 AndAlso Collections.Exists(Function(c) DirectCast(c, UserDataBase).ContentMissingExists) + End Get + End Property Friend ReadOnly Property Count As Integer Implements ICollection(Of IUserData).Count, IMyEnumerator(Of IUserData).MyEnumeratorCount Get If Collections Is Nothing Then @@ -285,8 +290,8 @@ Namespace API Friend Overrides Sub UpdateUserInformation() If Count > 0 Then Collections.ForEach(Sub(c) c.UpdateUserInformation()) End Sub - Friend Overrides Sub LoadContentInformation() - If Count > 0 Then Collections.ForEach(Sub(c) DirectCast(c, UserDataBase).LoadContentInformation()) + Friend Overrides Sub LoadContentInformation(Optional ByVal Force As Boolean = False) + If Count > 0 Then Collections.ForEach(Sub(c) DirectCast(c, UserDataBase).LoadContentInformation(Force)) End Sub Protected Overrides Sub LoadUserInformation_OptionalFields(ByRef Container As XmlFile, ByVal Loading As Boolean) End Sub @@ -309,8 +314,6 @@ Namespace API End Sub Protected Overrides Sub DownloadDataF(ByVal Token As CancellationToken) End Sub - Protected Overrides Sub ReparseVideo(ByVal Token As CancellationToken) - End Sub Protected Overrides Sub DownloadContent(ByVal Token As CancellationToken) End Sub Protected Overrides Function DownloadingException(ByVal ex As Exception, ByVal Message As String, Optional ByVal FromPE As Boolean = False) As Integer @@ -359,7 +362,7 @@ Namespace API ConsolidateScripts() .UpdateUserInformation() End If - ImageHandler(_Item, False) + MainFrameObj.ImageHandler(_Item, False) AddRemoveBttDeleteHandler(.Self, True) AddHandler .Self.UserUpdated, AddressOf User_OnUserUpdated End With @@ -454,7 +457,7 @@ Namespace API Return False Else DirectCast(_Item, UserDataBase).MoveFiles(String.Empty) - ImageHandler(_Item) + MainFrameObj.ImageHandler(_Item) AddRemoveBttDeleteHandler(_Item, False) RaiseEvent OnUserRemoved(_Item) Return Collections.Remove(_Item) @@ -469,7 +472,7 @@ Namespace API Settings.Users.Remove(Me) Collections.ForEach(Sub(c) c.Delete()) Downloader.UserRemove(Me) - ImageHandler(Me, False) + MainFrameObj.ImageHandler(Me, False) Collections.ListClearDispose Dispose(False) f.Delete(SFO.Path, SFODelete.EmptyOnly + Settings.DeleteMode, EDP.SendInLog) @@ -487,12 +490,12 @@ Namespace API Settings.Users.Remove(Me) Collections.ForEach(Sub(c) c.MoveFiles(String.Empty) - ImageHandler(c) + MainFrameObj.ImageHandler(c) End Sub) Collections.Clear() f.Delete(SFO.Path, SFODelete.Default + Settings.DeleteMode, EDP.SendInLog) Downloader.UserRemove(Me) - ImageHandler(Me, False) + MainFrameObj.ImageHandler(Me, False) Dispose(False) Return 3 Else @@ -511,7 +514,7 @@ Namespace API Dim RemoveMeIfNull As Action = Sub() If Count = 0 Then Settings.Users.Remove(Me) - ImageHandler(Me, False) + MainFrameObj.ImageHandler(Me, False) RaiseEvent OnCollectionSelfRemoved(Me) Dispose(False) End If diff --git a/SCrawler/Channels/ChannelViewForm.vb b/SCrawler/Channels/ChannelViewForm.vb index 0625696..5a047a4 100644 --- a/SCrawler/Channels/ChannelViewForm.vb +++ b/SCrawler/Channels/ChannelViewForm.vb @@ -142,7 +142,7 @@ Friend Class ChannelViewForm : Implements IChannelLimits Friend Sub New() InitializeComponent() MyDefs = New DefaultFormOptions - CProgress = New MyProgress(ToolbarBOTTOM, PR_CN, LBL_STATUS, "Downloading data") With {.PerformMod = 10, .DropCurrentProgressOnTotalChange = False} + CProgress = New MyProgress(ToolbarBOTTOM, PR_CN, LBL_STATUS, "Downloading data") With {.PerformMod = 10, .ResetProgressOnMaximumChanges = False} CProvider = New ANumbers With {.FormatOptions = ANumbers.Options.GroupIntegral} LimitProvider = New ADateTime("dd.MM.yyyy HH:mm") PendingUsers = New List(Of PendingUser) @@ -330,9 +330,9 @@ Friend Class ChannelViewForm : Implements IChannelLimits If Not TokenSource Is Nothing OrElse Not HOST.Source.Available(Plugin.ISiteSettings.Download.Channel, False) Then Exit Sub Dim InvokeToken As Action = Sub() If TokenSource Is Nothing Then - CProgress.TotalCount = 0 - CProgress.CurrentCounter = 0 - CProgress.Enabled = True + CProgress.Maximum = 0 + CProgress.Value = 0 + CProgress.Visible = True TokenSource = New CancellationTokenSource Token = TokenSource.Token BTT_DOWNLOAD.Enabled = False @@ -389,7 +389,7 @@ Friend Class ChannelViewForm : Implements IChannelLimits Finally If Not TokenSource Is Nothing AndAlso Not Settings.Channels.Downloading Then TokenSource = Nothing - CProgress.Enabled = False + CProgress.Visible = False BTT_DOWNLOAD.Enabled = True BTT_STOP.Enabled = False _CollectionDownloading = False diff --git a/SCrawler/Content/Icons/RSSIcon.ico b/SCrawler/Content/Icons/RSSIcon.ico new file mode 100644 index 0000000..8b28f09 Binary files /dev/null and b/SCrawler/Content/Icons/RSSIcon.ico differ diff --git a/SCrawler/Content/Icons/RegGifsIcon.ico b/SCrawler/Content/Icons/RedGifsIcon.ico similarity index 100% rename from SCrawler/Content/Icons/RegGifsIcon.ico rename to SCrawler/Content/Icons/RedGifsIcon.ico diff --git a/SCrawler/Content/Pictures/RSSPic.png b/SCrawler/Content/Pictures/RSSPic.png new file mode 100644 index 0000000..ad6c561 Binary files /dev/null and b/SCrawler/Content/Pictures/RSSPic.png differ diff --git a/SCrawler/Content/Pictures/RegGifsPic32.png b/SCrawler/Content/Pictures/RedGifsPic32.png similarity index 100% rename from SCrawler/Content/Pictures/RegGifsPic32.png rename to SCrawler/Content/Pictures/RedGifsPic32.png diff --git a/SCrawler/Download/ActiveDownloadingProgress.vb b/SCrawler/Download/ActiveDownloadingProgress.vb index 374f78d..b8dd0c8 100644 --- a/SCrawler/Download/ActiveDownloadingProgress.vb +++ b/SCrawler/Download/ActiveDownloadingProgress.vb @@ -23,8 +23,8 @@ Namespace DownloadObjects Friend Sub New() InitializeComponent() JobsList = New List(Of DownloadProgress) - AddHandler Downloader.OnReconfigured, AddressOf Downloader_OnReconfigured - Downloader_OnReconfigured() + AddHandler Downloader.Reconfigured, AddressOf Downloader_Reconfigured + Downloader_Reconfigured() End Sub Private Sub ActiveDownloadingProgress_Load(sender As Object, e As EventArgs) Handles Me.Load MyView = New FormsView(Me) @@ -37,7 +37,7 @@ Namespace DownloadObjects e.Cancel = True Hide() End Sub - Private Sub Downloader_OnReconfigured() + Private Sub Downloader_Reconfigured() Const RowHeight% = 30 Dim a As Action = Sub() With TP_MAIN @@ -58,7 +58,7 @@ Namespace DownloadObjects .RowStyles.Add(New RowStyle(SizeType.Absolute, RowHeight)) .RowCount += 1 JobsList.Add(New DownloadProgress(j)) - AddHandler JobsList.Last.OnTotalCountChange, AddressOf Jobs_OnTotalCountChange + AddHandler JobsList.Last.ProgressMaximumChanged, AddressOf Jobs_ProgressMaximumChanged .Controls.Add(JobsList.Last.Get, 0, .RowStyles.Count - 1) End With Next @@ -74,11 +74,11 @@ Namespace DownloadObjects End Sub If TP_MAIN.InvokeRequired Then TP_MAIN.Invoke(a) Else a.Invoke End Sub - Private Sub Jobs_OnTotalCountChange() + Private Sub Jobs_ProgressMaximumChanged() If JobsList.Count > 0 And Not DisableProgressChange Then - MainProgress.TotalCount = JobsList.Sum(Function(j) CLng(j.Job.Progress.TotalCount)) - MainProgress.CurrentCounter = Math.Max(JobsList.Sum(Function(j) CLng(j.Job.Progress.CurrentCounter)) - 1, 0) - If MainProgress.CurrentCounter > 0 Then MainProgress.Perform() + MainProgress.Maximum = JobsList.Sum(Function(j) CLng(j.Job.Progress.Maximum)) + MainProgress.Value = Math.Max(JobsList.Sum(Function(j) CLng(j.Job.Progress.Value)) - 1, 0) + If MainProgress.Value > 0 Then MainProgress.Perform() End If End Sub End Class diff --git a/SCrawler/Download/AutoDownloader.vb b/SCrawler/Download/AutoDownloader.vb index 38b08e7..a022d58 100644 --- a/SCrawler/Download/AutoDownloader.vb +++ b/SCrawler/Download/AutoDownloader.vb @@ -16,7 +16,6 @@ Imports SCrawler.API Imports SCrawler.API.Base Namespace DownloadObjects Friend Class AutoDownloader : Inherits GroupParameters : Implements IEContainerProvider - Friend Event UserFind(ByVal Key As String, ByVal Activate As Boolean) Friend Shared ReadOnly Property CachePath As SFile Get Return "_Cache\" @@ -174,6 +173,7 @@ Namespace DownloadObjects Private Const Name_ShowNotifications As String = "Notify" Private Const Name_ShowPictureDown As String = "ShowDownloadedPicture" Private Const Name_ShowPictureUser As String = "ShowUserPicture" + Private Const Name_ShowSimpleNotification As String = "ShowSimpleNotification" #End Region #Region "Declarations" Friend Property Source As Scheduler @@ -193,6 +193,7 @@ Namespace DownloadObjects Friend Property ShowNotifications As Boolean = True Friend Property ShowPictureDownloaded As Boolean = True Friend Property ShowPictureUser As Boolean = True + Friend Property ShowSimpleNotification As Boolean = False #Region "Date" Private ReadOnly LastDownloadDateXML As Date? = Nothing Private _LastDownloadDate As Date = Now.AddYears(-1) @@ -279,6 +280,7 @@ Namespace DownloadObjects ShowNotifications = x.Value(Name_ShowNotifications).FromXML(Of Boolean)(True) ShowPictureDownloaded = x.Value(Name_ShowPictureDown).FromXML(Of Boolean)(True) ShowPictureUser = x.Value(Name_ShowPictureUser).FromXML(Of Boolean)(True) + ShowSimpleNotification = x.Value(Name_ShowSimpleNotification).FromXML(Of Boolean)(False) LastDownloadDateXML = AConvert(Of Date)(x.Value(Name_LastDownloadDate), DateProvider, Nothing) If LastDownloadDateXML.HasValue Then LastDownloadDate = LastDownloadDateXML.Value @@ -321,6 +323,7 @@ Namespace DownloadObjects New EContainer(Name_ShowNotifications, ShowNotifications.BoolToInteger), New EContainer(Name_ShowPictureDown, ShowPictureDownloaded.BoolToInteger), New EContainer(Name_ShowPictureUser, ShowPictureUser.BoolToInteger), + New EContainer(Name_ShowSimpleNotification, ShowSimpleNotification.BoolToInteger), New EContainer(Name_LastDownloadDate, CStr(AConvert(Of String)(If(LastDownloadDateXML.HasValue Or _LastDownloadDateChanged, CObj(LastDownloadDate), Nothing), DateProvider, String.Empty))) } @@ -381,12 +384,18 @@ Namespace DownloadObjects Dim users As New List(Of IUserData) Dim GName$ Dim i% + Dim DownloadedUsersCount% = 0 Dim l As New ListAddParams(LAP.IgnoreICopier + LAP.NotContainsOnly) + Dim simple As Boolean = ShowSimpleNotification And ShowNotifications Dim notify As Action = Sub() With Downloader.Downloaded If ShowNotifications And .Count > 0 Then .ForEach(Sub(ByVal u As IUserData) If Keys.Contains(u.Key) Then - ShowNotification(u) + If simple Then + DownloadedUsersCount += 1 + Else + ShowNotification(u) + End If Keys.Remove(u.Key) End If End Sub) @@ -426,6 +435,8 @@ Namespace DownloadObjects While .Working Or .Count > 0 : notify.Invoke() : Thread.Sleep(200) : End While .AutoDownloaderWorking = False notify.Invoke + If simple And DownloadedUsersCount > 0 Then _ + MainFrameObj.ShowNotification($"{DownloadedUsersCount} user(s) downloaded with scheduler plan '{Name}'", $"Scheduler plan '{Name}'") End With End If Catch ex As Exception @@ -443,14 +454,14 @@ Namespace DownloadObjects If i >= 0 Then UserKeys(i).ShowNotification() Else - UserKeys.Add(New NotifiedUser(k, TDownloader.GetUserFromMainCollection(u), Me)) + UserKeys.Add(New NotifiedUser(k, Settings.GetUser(u), Me)) UserKeys.Last.ShowNotification() End If End Sub Friend Function NotificationClicked(ByVal Key As String) As Boolean Dim i% = UserKeys.IndexOf(Key) If i >= 0 Then - RaiseEvent UserFind(UserKeys(i).IUserDataKey, UserKeys(i).Open(Key)) + MainFrameObj.FocusUser(UserKeys(i).IUserDataKey, UserKeys(i).Open(Key)) Return True Else Return False diff --git a/SCrawler/Download/AutoDownloaderEditorForm.Designer.vb b/SCrawler/Download/AutoDownloaderEditorForm.Designer.vb index cf361ed..1fd4f22 100644 --- a/SCrawler/Download/AutoDownloaderEditorForm.Designer.vb +++ b/SCrawler/Download/AutoDownloaderEditorForm.Designer.vb @@ -45,6 +45,7 @@ Namespace DownloadObjects Me.CH_NOTIFY = New System.Windows.Forms.CheckBox() Me.CH_SHOW_PIC = New System.Windows.Forms.CheckBox() Me.CH_SHOW_PIC_USER = New System.Windows.Forms.CheckBox() + Me.CH_NOTIFY_SIMPLE = New System.Windows.Forms.CheckBox() CONTAINER_MAIN = New System.Windows.Forms.ToolStripContainer() TP_MODE = New System.Windows.Forms.TableLayoutPanel() TP_NOTIFY = New System.Windows.Forms.TableLayoutPanel() @@ -119,7 +120,7 @@ Namespace DownloadObjects Me.TXT_GROUPS.Location = New System.Drawing.Point(4, 140) Me.TXT_GROUPS.Name = "TXT_GROUPS" Me.TXT_GROUPS.Size = New System.Drawing.Size(468, 22) - Me.TXT_GROUPS.TabIndex = 2 + Me.TXT_GROUPS.TabIndex = 1 ' 'TP_MODE ' @@ -220,7 +221,7 @@ Namespace DownloadObjects Me.TXT_TIMER.Location = New System.Drawing.Point(4, 195) Me.TXT_TIMER.Name = "TXT_TIMER" Me.TXT_TIMER.Size = New System.Drawing.Size(468, 22) - Me.TXT_TIMER.TabIndex = 2 + Me.TXT_TIMER.TabIndex = 3 ' 'LBL_LAST_TIME_UP ' @@ -230,7 +231,7 @@ Namespace DownloadObjects Me.LBL_LAST_TIME_UP.Location = New System.Drawing.Point(4, 250) Me.LBL_LAST_TIME_UP.Name = "LBL_LAST_TIME_UP" Me.LBL_LAST_TIME_UP.Size = New System.Drawing.Size(468, 25) - Me.LBL_LAST_TIME_UP.TabIndex = 4 + Me.LBL_LAST_TIME_UP.TabIndex = 5 Me.LBL_LAST_TIME_UP.Text = "Last download date: " Me.LBL_LAST_TIME_UP.TextAlign = System.Drawing.ContentAlignment.TopCenter ' @@ -251,27 +252,28 @@ Namespace DownloadObjects Me.NUM_DELAY.NumberMaximum = New Decimal(New Integer() {1440, 0, 0, 0}) Me.NUM_DELAY.NumberUpDownAlign = System.Windows.Forms.LeftRightAlignment.Left Me.NUM_DELAY.Size = New System.Drawing.Size(468, 22) - Me.NUM_DELAY.TabIndex = 3 + Me.NUM_DELAY.TabIndex = 4 Me.NUM_DELAY.Text = "0" ' 'TP_NOTIFY ' - TP_NOTIFY.ColumnCount = 3 - TP_NOTIFY.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Percent, 33.33333!)) - TP_NOTIFY.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Percent, 33.33333!)) - TP_NOTIFY.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Percent, 33.33333!)) + TP_NOTIFY.ColumnCount = 4 + TP_NOTIFY.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Percent, 25.0!)) + TP_NOTIFY.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Percent, 25.0!)) + TP_NOTIFY.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Percent, 25.0!)) + TP_NOTIFY.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Percent, 25.0!)) TP_NOTIFY.Controls.Add(Me.CH_NOTIFY, 0, 0) - TP_NOTIFY.Controls.Add(Me.CH_SHOW_PIC, 1, 0) - TP_NOTIFY.Controls.Add(Me.CH_SHOW_PIC_USER, 2, 0) + TP_NOTIFY.Controls.Add(Me.CH_SHOW_PIC, 2, 0) + TP_NOTIFY.Controls.Add(Me.CH_SHOW_PIC_USER, 3, 0) + TP_NOTIFY.Controls.Add(Me.CH_NOTIFY_SIMPLE, 1, 0) TP_NOTIFY.Dock = System.Windows.Forms.DockStyle.Fill TP_NOTIFY.Location = New System.Drawing.Point(1, 166) TP_NOTIFY.Margin = New System.Windows.Forms.Padding(0) TP_NOTIFY.Name = "TP_NOTIFY" TP_NOTIFY.RowCount = 1 TP_NOTIFY.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Percent, 100.0!)) - TP_NOTIFY.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!)) TP_NOTIFY.Size = New System.Drawing.Size(474, 25) - TP_NOTIFY.TabIndex = 1 + TP_NOTIFY.TabIndex = 2 ' 'CH_NOTIFY ' @@ -279,7 +281,7 @@ Namespace DownloadObjects Me.CH_NOTIFY.Dock = System.Windows.Forms.DockStyle.Fill Me.CH_NOTIFY.Location = New System.Drawing.Point(3, 3) Me.CH_NOTIFY.Name = "CH_NOTIFY" - Me.CH_NOTIFY.Size = New System.Drawing.Size(152, 19) + Me.CH_NOTIFY.Size = New System.Drawing.Size(112, 19) Me.CH_NOTIFY.TabIndex = 0 Me.CH_NOTIFY.Text = "Show notifications" TT_MAIN.SetToolTip(Me.CH_NOTIFY, "Show notification when some data has been downloaded") @@ -289,11 +291,11 @@ Namespace DownloadObjects ' Me.CH_SHOW_PIC.AutoSize = True Me.CH_SHOW_PIC.Dock = System.Windows.Forms.DockStyle.Fill - Me.CH_SHOW_PIC.Location = New System.Drawing.Point(161, 3) + Me.CH_SHOW_PIC.Location = New System.Drawing.Point(239, 3) Me.CH_SHOW_PIC.Name = "CH_SHOW_PIC" - Me.CH_SHOW_PIC.Size = New System.Drawing.Size(152, 19) - Me.CH_SHOW_PIC.TabIndex = 1 - Me.CH_SHOW_PIC.Text = "Show download picture" + Me.CH_SHOW_PIC.Size = New System.Drawing.Size(112, 19) + Me.CH_SHOW_PIC.TabIndex = 2 + Me.CH_SHOW_PIC.Text = "Image" TT_MAIN.SetToolTip(Me.CH_SHOW_PIC, "Show downloaded image in notification") Me.CH_SHOW_PIC.UseVisualStyleBackColor = True ' @@ -301,14 +303,26 @@ Namespace DownloadObjects ' Me.CH_SHOW_PIC_USER.AutoSize = True Me.CH_SHOW_PIC_USER.Dock = System.Windows.Forms.DockStyle.Fill - Me.CH_SHOW_PIC_USER.Location = New System.Drawing.Point(319, 3) + Me.CH_SHOW_PIC_USER.Location = New System.Drawing.Point(357, 3) Me.CH_SHOW_PIC_USER.Name = "CH_SHOW_PIC_USER" - Me.CH_SHOW_PIC_USER.Size = New System.Drawing.Size(152, 19) - Me.CH_SHOW_PIC_USER.TabIndex = 2 - Me.CH_SHOW_PIC_USER.Text = "Show user picture" + Me.CH_SHOW_PIC_USER.Size = New System.Drawing.Size(114, 19) + Me.CH_SHOW_PIC_USER.TabIndex = 3 + Me.CH_SHOW_PIC_USER.Text = "User icon" TT_MAIN.SetToolTip(Me.CH_SHOW_PIC_USER, "Show user image in notification") Me.CH_SHOW_PIC_USER.UseVisualStyleBackColor = True ' + 'CH_NOTIFY_SIMPLE + ' + Me.CH_NOTIFY_SIMPLE.AutoSize = True + Me.CH_NOTIFY_SIMPLE.Dock = System.Windows.Forms.DockStyle.Fill + Me.CH_NOTIFY_SIMPLE.Location = New System.Drawing.Point(121, 3) + Me.CH_NOTIFY_SIMPLE.Name = "CH_NOTIFY_SIMPLE" + Me.CH_NOTIFY_SIMPLE.Size = New System.Drawing.Size(112, 19) + Me.CH_NOTIFY_SIMPLE.TabIndex = 1 + Me.CH_NOTIFY_SIMPLE.Text = "Simple" + TT_MAIN.SetToolTip(Me.CH_NOTIFY_SIMPLE, resources.GetString("CH_NOTIFY_SIMPLE.ToolTip")) + Me.CH_NOTIFY_SIMPLE.UseVisualStyleBackColor = True + ' 'AutoDownloaderEditorForm ' Me.AutoScaleDimensions = New System.Drawing.SizeF(6.0!, 13.0!) @@ -347,11 +361,12 @@ Namespace DownloadObjects Private WithEvents OPT_SPEC As RadioButton Private WithEvents OPT_DISABLED As RadioButton Private WithEvents CH_NOTIFY As CheckBox - Friend WithEvents TXT_TIMER As PersonalUtilities.Forms.Controls.TextBoxExtended + Private WithEvents TXT_TIMER As PersonalUtilities.Forms.Controls.TextBoxExtended Private WithEvents OPT_GROUP As RadioButton Private WithEvents LBL_LAST_TIME_UP As Label Private WithEvents NUM_DELAY As PersonalUtilities.Forms.Controls.TextBoxExtended Private WithEvents CH_SHOW_PIC As CheckBox Private WithEvents CH_SHOW_PIC_USER As CheckBox + Private WithEvents CH_NOTIFY_SIMPLE As CheckBox End Class End Namespace \ No newline at end of file diff --git a/SCrawler/Download/AutoDownloaderEditorForm.resx b/SCrawler/Download/AutoDownloaderEditorForm.resx index f3a0988..d121f4a 100644 --- a/SCrawler/Download/AutoDownloaderEditorForm.resx +++ b/SCrawler/Download/AutoDownloaderEditorForm.resx @@ -231,6 +231,11 @@ False + + Show a simple notification instead of a user notification. +This means that if any user data has been downloaded with the plan, a simple notification will be shown with the number of users downloaded. +The 'Image' and 'User icon' parameters will be ignored. + AAABAAwAMDAQAAEABABoBgAAxgAAACAgEAABAAQA6AIAAC4HAAAYGBAAAQAEAOgBAAAWCgAAEBAQAAEA diff --git a/SCrawler/Download/AutoDownloaderEditorForm.vb b/SCrawler/Download/AutoDownloaderEditorForm.vb index 2b289a6..7a9dcd5 100644 --- a/SCrawler/Download/AutoDownloaderEditorForm.vb +++ b/SCrawler/Download/AutoDownloaderEditorForm.vb @@ -52,6 +52,7 @@ Namespace DownloadObjects If MyGroups.Count > 0 Then TXT_GROUPS.Text = MyGroups.ListToString If Settings.Groups.Count = 0 Then TXT_GROUPS.Clear() : TXT_GROUPS.Enabled = False CH_NOTIFY.Checked = .ShowNotifications + CH_NOTIFY_SIMPLE.Checked = .ShowSimpleNotification CH_SHOW_PIC.Checked = .ShowPictureDownloaded CH_SHOW_PIC_USER.Checked = .ShowPictureUser TXT_TIMER.Text = .Timer @@ -86,6 +87,7 @@ Namespace DownloadObjects .Groups.Clear() .Groups.ListAddList(MyGroups) .ShowNotifications = CH_NOTIFY.Checked + .ShowSimpleNotification = CH_NOTIFY_SIMPLE.Checked .ShowPictureDownloaded = CH_SHOW_PIC.Checked .ShowPictureUser = CH_SHOW_PIC_USER.Checked .Timer = AConvert(Of Integer)(TXT_TIMER.Text, AutoDownloader.DefaultTimer) @@ -120,17 +122,15 @@ Namespace DownloadObjects Private Sub OPT_GROUP_CheckedChanged(sender As Object, e As EventArgs) Handles OPT_GROUP.CheckedChanged ChangeEnabled() End Sub - Private Sub CH_NOTIFY_CheckedChanged(sender As Object, e As EventArgs) Handles CH_NOTIFY.CheckedChanged - ChangeEnabled() - End Sub - Private Sub ChangeEnabled() + Private Sub ChangeEnabled() Handles CH_NOTIFY.CheckedChanged, CH_NOTIFY_SIMPLE.CheckedChanged DEF_GROUP.Enabled = OPT_SPEC.Checked TXT_GROUPS.Enabled = OPT_GROUP.Checked TXT_TIMER.Enabled = Not OPT_DISABLED.Checked NUM_DELAY.Enabled = Not OPT_DISABLED.Checked CH_NOTIFY.Enabled = Not OPT_DISABLED.Checked - CH_SHOW_PIC.Enabled = CH_NOTIFY.Checked And Not OPT_DISABLED.Checked - CH_SHOW_PIC_USER.Enabled = CH_NOTIFY.Checked And Not OPT_DISABLED.Checked + CH_NOTIFY_SIMPLE.Enabled = CH_NOTIFY.Enabled And CH_NOTIFY.Checked + CH_SHOW_PIC.Enabled = CH_NOTIFY.Checked And Not OPT_DISABLED.Checked And Not CH_NOTIFY_SIMPLE.Checked + CH_SHOW_PIC_USER.Enabled = CH_NOTIFY.Checked And Not OPT_DISABLED.Checked And Not CH_NOTIFY_SIMPLE.Checked End Sub Private Sub NUM_DELAY_ActionOnButtonClick(ByVal Sender As ActionButton, ByVal e As EventArgs) Handles NUM_DELAY.ActionOnButtonClick If Sender.DefaultButton = ActionButton.DefaultButtons.Clear Then NUM_DELAY.Value = 0 diff --git a/SCrawler/Download/DownloadFeedForm.Designer.vb b/SCrawler/Download/DownloadFeedForm.Designer.vb new file mode 100644 index 0000000..e5d0a45 --- /dev/null +++ b/SCrawler/Download/DownloadFeedForm.Designer.vb @@ -0,0 +1,117 @@ +' Copyright (C) 2022 Andy +' This program is free software: you can redistribute it and/or modify +' it under the terms of the GNU General Public License as published by +' the Free Software Foundation, either version 3 of the License, or +' (at your option) any later version. +' +' This program is distributed in the hope that it will be useful, +' but WITHOUT ANY WARRANTY +Namespace DownloadObjects + + Partial Friend Class DownloadFeedForm : Inherits System.Windows.Forms.Form + + Protected Overrides Sub Dispose(ByVal disposing As Boolean) + Try + If disposing AndAlso components IsNot Nothing Then + components.Dispose() + End If + Finally + MyBase.Dispose(disposing) + End Try + End Sub + Private components As System.ComponentModel.IContainer + + Private Sub InitializeComponent() + Dim SEP_1 As System.Windows.Forms.ToolStripSeparator + Dim resources As System.ComponentModel.ComponentResourceManager = New System.ComponentModel.ComponentResourceManager(GetType(DownloadFeedForm)) + Me.ToolbarTOP = New System.Windows.Forms.ToolStrip() + Me.BTT_REFRESH = New System.Windows.Forms.ToolStripButton() + Me.BTT_CLEAR = New System.Windows.Forms.ToolStripButton() + Me.TP_DATA = New System.Windows.Forms.TableLayoutPanel() + SEP_1 = New System.Windows.Forms.ToolStripSeparator() + Me.ToolbarTOP.SuspendLayout() + Me.SuspendLayout() + ' + 'SEP_1 + ' + SEP_1.Name = "SEP_1" + SEP_1.Size = New System.Drawing.Size(6, 25) + ' + 'ToolbarTOP + ' + Me.ToolbarTOP.GripStyle = System.Windows.Forms.ToolStripGripStyle.Hidden + Me.ToolbarTOP.Items.AddRange(New System.Windows.Forms.ToolStripItem() {Me.BTT_REFRESH, Me.BTT_CLEAR, SEP_1}) + Me.ToolbarTOP.Location = New System.Drawing.Point(0, 0) + Me.ToolbarTOP.Name = "ToolbarTOP" + Me.ToolbarTOP.Size = New System.Drawing.Size(484, 25) + Me.ToolbarTOP.TabIndex = 0 + ' + 'BTT_REFRESH + ' + Me.BTT_REFRESH.Image = Global.SCrawler.My.Resources.Resources.Refresh + Me.BTT_REFRESH.ImageTransparentColor = System.Drawing.Color.Magenta + Me.BTT_REFRESH.Name = "BTT_REFRESH" + Me.BTT_REFRESH.Size = New System.Drawing.Size(66, 22) + Me.BTT_REFRESH.Text = "Refresh" + Me.BTT_REFRESH.ToolTipText = "Refresh data list" + ' + 'BTT_CLEAR + ' + Me.BTT_CLEAR.Image = Global.SCrawler.My.Resources.Resources.Delete + Me.BTT_CLEAR.ImageTransparentColor = System.Drawing.Color.Magenta + Me.BTT_CLEAR.Name = "BTT_CLEAR" + Me.BTT_CLEAR.Size = New System.Drawing.Size(54, 22) + Me.BTT_CLEAR.Text = "Clear" + Me.BTT_CLEAR.ToolTipText = "Clear data list" + ' + 'TP_DATA + ' + Me.TP_DATA.AutoScroll = True + Me.TP_DATA.ColumnCount = 1 + Me.TP_DATA.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Percent, 100.0!)) + Me.TP_DATA.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Absolute, 20.0!)) + Me.TP_DATA.Dock = System.Windows.Forms.DockStyle.Fill + Me.TP_DATA.Location = New System.Drawing.Point(0, 25) + Me.TP_DATA.Name = "TP_DATA" + Me.TP_DATA.RowCount = 11 + Me.TP_DATA.RowStyles.Add(New System.Windows.Forms.RowStyle()) + Me.TP_DATA.RowStyles.Add(New System.Windows.Forms.RowStyle()) + Me.TP_DATA.RowStyles.Add(New System.Windows.Forms.RowStyle()) + Me.TP_DATA.RowStyles.Add(New System.Windows.Forms.RowStyle()) + Me.TP_DATA.RowStyles.Add(New System.Windows.Forms.RowStyle()) + Me.TP_DATA.RowStyles.Add(New System.Windows.Forms.RowStyle()) + Me.TP_DATA.RowStyles.Add(New System.Windows.Forms.RowStyle()) + Me.TP_DATA.RowStyles.Add(New System.Windows.Forms.RowStyle()) + Me.TP_DATA.RowStyles.Add(New System.Windows.Forms.RowStyle()) + Me.TP_DATA.RowStyles.Add(New System.Windows.Forms.RowStyle()) + Me.TP_DATA.RowStyles.Add(New System.Windows.Forms.RowStyle()) + Me.TP_DATA.Size = New System.Drawing.Size(484, 436) + Me.TP_DATA.TabIndex = 1 + ' + 'DownloadFeedForm + ' + Me.AutoScaleDimensions = New System.Drawing.SizeF(6.0!, 13.0!) + Me.AutoScaleMode = System.Windows.Forms.AutoScaleMode.Font + Me.BackColor = System.Drawing.SystemColors.Window + Me.ClientSize = New System.Drawing.Size(484, 461) + Me.Controls.Add(Me.TP_DATA) + Me.Controls.Add(Me.ToolbarTOP) + Me.ForeColor = System.Drawing.SystemColors.WindowText + Me.Icon = CType(resources.GetObject("$this.Icon"), System.Drawing.Icon) + Me.KeyPreview = True + Me.MinimumSize = New System.Drawing.Size(300, 300) + Me.Name = "DownloadFeedForm" + Me.Text = "Download Feed" + Me.ToolbarTOP.ResumeLayout(False) + Me.ToolbarTOP.PerformLayout() + Me.ResumeLayout(False) + Me.PerformLayout() + + End Sub + + Private WithEvents ToolbarTOP As ToolStrip + Private WithEvents TP_DATA As TableLayoutPanel + Private WithEvents BTT_REFRESH As ToolStripButton + Private WithEvents BTT_CLEAR As ToolStripButton + End Class +End Namespace \ No newline at end of file diff --git a/SCrawler/Download/DownloadFeedForm.resx b/SCrawler/Download/DownloadFeedForm.resx new file mode 100644 index 0000000..53ad408 --- /dev/null +++ b/SCrawler/Download/DownloadFeedForm.resx @@ -0,0 +1,203 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + text/microsoft-resx + + + 2.0 + + + System.Resources.ResXResourceReader, System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 + + + System.Resources.ResXResourceWriter, System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 + + + False + + + 17, 17 + + + + + AAABAAEAICAAAAEAIACoEAAAFgAAACgAAAAgAAAAQAAAAAEAIAAAAAAAABAAABMLAAATCwAAAAAAAAAA + AAAZeOoAGXjqJBl46p4ZeOrtGXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl4 + 6v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq7Rl4 + 6p4ZeOokGXjqABl46iQZeOq+GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl4 + 6v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl4 + 6v8ZeOr/GXjq/xl46r4ZeOokGXjqnhl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl4 + 6v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl4 + 6v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46p4ZeOrrGXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GHfq/xh3 + 6v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8Yd+r/F3fq/xd36v8Xd+r/GHjq/xl46v8ZeOr/GHjq/xd3 + 6v8Xd+r/F3fq/xh36v8ZeOr/GXjq/xl46v8ZeOr/GXjq6xl46v4ZeOr/GXjq/xl46v8ZeOr/GHfq/xl4 + 6v8pguz/KYHr/xl46v8Yd+r/GXjq/xl46v8ZeOr/GXjq/yqC7P8yh+z/Mofs/zKH7P8mf+v/GXjq/xl4 + 6v8hfev/Mofs/zOH7P8zh+z/L4Xs/xt56v8ZeOr/GXjq/xl46v8ZeOr+GXjq/xl46v8ZeOr/GXjq/xh3 + 6v8ogev/j731/9Xm+//T5fv/ibr0/yV/6/8Yd+r/GXjq/xl46v8ceur/o8n3/+Pv/P/g7fz/4+78/3qx + 8/8Wdur/Fnbq/1id8P/e7Pz/4e38/+Pu/P/A2vn/KoLs/xh36v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl4 + 6v8ZeOr/GXjq/4289f/+/v/////////////9/v//hbf0/xh36v8ZeOr/GHjq/yN+6//J3/r///////// + ////////fLLz/xV26v8Vdur/bKjy//7+/////////////9Pl+/8ogev/GHfq/xl46v8ZeOr/GXjq/xl4 + 6v8ZeOr/GXjq/xh36v8ogev/0eT7///////////////////////K4Pr/JH/r/xh46v8Xd+r/N4rt/+Pv + /P////////////v9//9hovH/FXbq/xV26v+CtfT/////////////////xN36/yF96/8YeOr/GXjq/xl4 + 6v8ZeOr/GXjq/xl46v8ZeOr/GHfq/yeA6//P4/v//////////////////////8jf+v8jfuv/GHjq/xV2 + 6v9npvH/+/3/////////////6vP9/z+P7v8Xd+r/GXjq/6PJ9/////////////////+rzvf/Gnnq/xl4 + 6v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GHjq/4W39P/8/f/////////////6/P//fLLz/xh3 + 6v8YeOr/IHzr/7bU+P/////////////////C2/n/In3r/xh36v8mgOv/zOH6/////////////////4e4 + 9P8Wdur/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8Yd+r/JH7r/4C18//I3/r/x976/3uy + 8/8hfev/GHjq/xZ26v9lpfH/9fn+/////////////////32z8/8Wdur/Fnbq/0qU7v/w9v7///////// + ///4+/7/Wp7w/xV26v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8YeOr/GHfq/yN+ + 6/8ifev/F3fq/xh46v8Wdur/QI/u/9jo+//////////////////a6vz/NYnt/xd36v8Xd+r/i7v0//// + /////////////9zr/P8yhuz/F3fq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl4 + 6v8ZeOr/GHjq/xh46v8YeOr/Fnbq/z+O7f/I3/r//////////////////P3//32z8/8Xd+r/F3fq/zCF + 7P/V5vv/////////////////pMr3/xp56v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl4 + 6v8ZeOr/GXjq/xh46v8Xd+r/FXbq/x976/9jo/H/1+f7//////////////////////+41fj/JoDr/xh3 + 6v8Xd+r/frPz//3+//////////////b6/v9anvD/Fnbq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl4 + 6v8ZeOr/GXjq/xl46v8beer/IX3r/zSI7P9jpPH/s9P4//T5/v//////////////////////0uX7/z2N + 7f8Xd+r/Fnfq/z2N7f/d6/z/////////////////wNr5/yR+6/8YeOr/GXjq/xl46v8ZeOr/GXjq/xl4 + 6v8ZeOr/GXjq/xl46v8Yd+r/JoDr/53F9v/G3vr/4e38//n8/v///////////////////////////9Pl + +/9IlO7/Fnfq/xh36v8ifev/rc/4//////////////////b6/v9jo/H/Fnbq/xl46v8ZeOr/GXjq/xl4 + 6v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xh36v8thOz/2+r8//////////////////////////////////z9 + //+61/n/Po7t/xZ36v8YeOr/G3nq/4m69P/7/P//////////////////r9D4/yF86/8YeOr/GXjq/xl4 + 6v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GHfq/y2E7P/b6vz///////////////////////7+ + ///e7Pz/gLTz/yeA6/8Xd+r/GHfq/xt56v97svP/9Pj+/////////////////+Dt/P9DkO7/Fnfq/xl4 + 6v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8Yd+r/LYTs/9vq/P///////P3//+31 + /f/G3vr/gbX0/ziK7f8Xd+r/GHfq/xd36v8hfev/h7j0//T4/v/////////////////0+P7/bKny/xd3 + 6v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xh46v8kfuv/e7Hz/4G1 + 9P9mpfH/Q5Hu/yR+6/8Wdur/F3fq/xh36v8Wdur/OYvt/6rN9//5/P7/////////////////+vz//4q7 + 9P8ceur/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl4 + 6v8Wdur/FXbq/xV26v8Wdur/GHfq/xZ26v8Wdur/K4Ps/3ev8//a6fz///////////////////////r8 + //+VwfX/IHzr/xh36v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl4 + 6v8ZeOr/GXjq/xZ26v8Vdur/FXbq/xd36v8ifev/QpDu/4K29P/Q4/v/+/3///////////////////// + ///1+f7/jLz1/yF96/8Yd+r/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl4 + 6v8ZeOr/GXjq/xl46v8ee+r/T5fv/2Oj8f94sPP/msT2/8Td+v/s9P3//v7///////////////////// + ////////4+/8/3Cr8v8ceur/GHfq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl4 + 6v8ZeOr/GXjq/xl46v8ZeOr/GHfq/yyD7P/V5vv//v////////////////////////////////////// + ////////+Pv+/7XU+P9Gku7/F3fq/xh46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl4 + 6v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8Yd+r/LYPs/9rp/P////////////////////////////// + ////////+Pv+/8fe+v9pp/H/In7r/xZ26v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl4 + 6v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xh36v8thOz/2un8//////////////////// + ///7/f//4+/8/63P+P9io/H/J4Dr/xZ26v8YeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl4 + 6v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GHfq/yqC7P+/2vn/3Ov8/83i + +v+11Pj/kb71/2Ok8f83iu3/HHrq/xZ26v8Yd+r/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl4 + 6v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr+GXjq/xl46v8ZeOr/G3nq/y6E + 7P8thOz/JX/r/xx66v8Xd+r/FXbq/xd36v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl4 + 6v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/hl46usZeOr/GXjq/xl4 + 6v8ZeOr/GHfq/xh36v8YeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl4 + 6v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOrrGXjqnhl4 + 6v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl4 + 6v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl4 + 6p4ZeOokGXjqvhl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl4 + 6v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl4 + 6v8ZeOq+GXjqJBl46gAZeOokGXjqnxl46u0ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl4 + 6v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl46v8ZeOr/GXjq/xl4 + 6v8ZeOrtGXjqnhl46iQZeOoAgAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA + AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA + AAAAAAAAAAAAAAAAAAAAAAAAAAAAAIAAAAE= + + + \ No newline at end of file diff --git a/SCrawler/Download/DownloadFeedForm.vb b/SCrawler/Download/DownloadFeedForm.vb new file mode 100644 index 0000000..3fb9a80 --- /dev/null +++ b/SCrawler/Download/DownloadFeedForm.vb @@ -0,0 +1,326 @@ +' Copyright (C) 2022 Andy +' This program is free software: you can redistribute it and/or modify +' it under the terms of the GNU General Public License as published by +' the Free Software Foundation, either version 3 of the License, or +' (at your option) any later version. +' +' This program is distributed in the hope that it will be useful, +' but WITHOUT ANY WARRANTY +Imports System.ComponentModel +Imports PersonalUtilities.Forms +Imports PersonalUtilities.Forms.Toolbars +Imports PersonalUtilities.Tools +Imports RCI = PersonalUtilities.Forms.Toolbars.RangeSwitcherToolbar.ControlItem +Imports UserMediaD = SCrawler.DownloadObjects.TDownloader.UserMediaD +Namespace DownloadObjects + Friend Class DownloadFeedForm +#Region "Declarations" + Private WithEvents MyDefs As DefaultFormOptions + Private WithEvents MyRange As RangeSwitcherToolbar(Of UserMediaD) + Private ReadOnly DataList As List(Of UserMediaD) + Private WithEvents BTT_DELETE_SELECTED As ToolStripButton + Private WithEvents LBL_FILES As ToolStripLabel + Private DataRows As Integer = 10 + Private DataColumns As Integer = 1 + Private FeedEndless As Boolean = False +#End Region +#Region "Initializer" + Friend Sub New() + InitializeComponent() + MyDefs = New DefaultFormOptions(Me, Settings.Design) + MyRange = New RangeSwitcherToolbar(Of UserMediaD)(ToolbarTOP) + DataList = New List(Of UserMediaD) + LBL_FILES = New ToolStripLabel With {.Text = String.Empty, .AutoToolTip = False, .ToolTipText = String.Empty} + BTT_DELETE_SELECTED = New ToolStripButton With { + .Text = "Delete selected", + .AutoToolTip = True, + .ToolTipText = "Delete marked files", + .Image = My.Resources.Delete, + .DisplayStyle = ToolStripItemDisplayStyle.ImageAndText + } + End Sub +#End Region +#Region "Form handlers" + Private Sub DownloadFeedForm_Load(sender As Object, e As EventArgs) Handles Me.Load + With MyDefs + .MyViewInitialize() + With MyRange + .AutoToolTip = True + .ButtonKey(RCI.Previous) = Keys.F3 + .ButtonKey(RCI.Next) = Keys.F4 + .AddThisToolbar() + End With + ToolbarTOP.Items.AddRange({New ToolStripSeparator, BTT_DELETE_SELECTED, New ToolStripSeparator, LBL_FILES}) + UpdateSettings() + RefillList() + .EndLoaderOperations(False) + End With + End Sub + Private Sub DownloadFeedForm_Closing(sender As Object, e As CancelEventArgs) Handles Me.Closing + e.Cancel = True + Hide() + End Sub + Private Sub DownloadFeedForm_Disposed(sender As Object, e As EventArgs) Handles Me.Disposed + ClearTable() + MyRange.Dispose() + BTT_CLEAR.Dispose() + DataList.Clear() + LBL_FILES.Dispose() + End Sub + Private Sub DownloadFeedForm_KeyDown(sender As Object, e As KeyEventArgs) Handles Me.KeyDown + If e.KeyCode = Keys.F5 Then RefillList() : e.Handled = True + End Sub +#End Region +#Region "Settings" + Friend Sub UpdateSettings() + With Settings + Dim c% = .FeedDataRows * .FeedDataColumns + Dim rangeChanged As Boolean = Not c = DataRows * DataColumns + DataRows = .FeedDataRows + DataColumns = .FeedDataColumns + FeedEndless = .FeedEndless + If rangeChanged Then + ClearTable() + ControlInvoke(TP_DATA, Sub() + With TP_DATA + .RowStyles.Clear() + .RowCount = 0 + .ColumnStyles.Clear() + .ColumnCount = 0 + Dim i% + Dim p% = IIf(DataColumns = 1, 100, 50) + For i = 0 To DataColumns - 1 : .ColumnStyles.Add(New ColumnStyle(SizeType.Percent, p)) : Next + .ColumnCount = .ColumnStyles.Count + For i = 0 To DataRows - 1 : .RowStyles.Add(New RowStyle(SizeType.Absolute, 0)) : Next + .RowCount = .RowStyles.Count + .HorizontalScroll.Visible = False + End With + End Sub) + End If + MyRange.HandlersSuspended = True + MyRange.Limit = c + MyRange.HandlersSuspended = False + If Not MyDefs.Initializing And rangeChanged Then RefillList() + End With + End Sub +#End Region +#Region "Refill" + Friend Sub Downloader_FilesChanged(ByVal Added As Boolean) + ControlInvoke(ToolbarTOP, LBL_FILES, Sub() LBL_FILES.Text = IIf(Added, "New files found", "Some files have been removed")) + LBL_FILES.ControlChangeColor(ToolbarTOP, Added, False) + End Sub + Private Sub RefillList() Handles BTT_REFRESH.Click + DataPopulated = False + DataList.ListAddList(Downloader.Files, LAP.ClearBeforeAdd, LAP.NotContainsOnly) + MyRange.Source = DataList + ControlInvoke(ToolbarTOP, LBL_FILES, Sub() LBL_FILES.Text = String.Empty) + LBL_FILES.ControlDropColor(ToolbarTOP) + If DataList.Count = 0 Then + ClearTable() + ElseIf Not DataPopulated Then + MyRange_IndexChanged(MyRange, Nothing) + End If + End Sub + Private Sub BTT_CLEAR_Click(sender As Object, e As EventArgs) Handles BTT_CLEAR.Click + Downloader.Files.Clear() + ClearTable() + RefillList() + End Sub +#End Region +#Region "Data" + Private Sub BTT_DELETE_SELECTED_Click(sender As Object, e As EventArgs) Handles BTT_DELETE_SELECTED.Click + Const MsgTitle$ = "Deleting marked files" + Try + Dim c As IEnumerable(Of FeedMedia) = ControlInvoke(TP_DATA, Function() If(TP_DATA.Controls.Count > 0, TP_DATA.Controls.ToObjectsList.Cast(Of FeedMedia)().Where(Function(f) f.Checked), New FeedMedia() {})) + If c.ListExists Then + If MsgBoxE({$"Are you sure you want to delete {c.Count} file(s)?", MsgTitle}, vbExclamation,,, {"Process", "Cancel"}) = 0 Then + Dim indx% = MyRange.CurrentIndex + Dim d% = 0 + ControlInvoke(TP_DATA, Sub() + With TP_DATA + For Each fm As FeedMedia In c + If fm.DeleteFile(True) Then + d += 1 + DataList.RemoveAll(Function(dd) dd.Data.File = fm.File) + End If + Next + End With + End Sub) + If d > 0 Then + If DataList.Count > 0 Then + MyRange.HandlersSuspended = True + MyRange.Source = DataList + If indx.ValueBetween(0, MyRange.Count - 1) Then MyRange.CurrentIndex = indx + MyRange.HandlersSuspended = False + DirectCast(MyRange.Switcher, RangeSwitcher(Of UserMediaD)).PerformIndexChanged() + Else + RefillList() + End If + End If + MsgBoxE({$"{d}/{c.Count} file(s) deleted", MsgTitle}) + Else + MsgBoxE({"Operation canceled", MsgTitle}) + End If + Else + MsgBoxE({"No files selected", MsgTitle}, vbExclamation) + End If + Catch ex As Exception + ErrorsDescriber.Execute(EDP.LogMessageValue, ex, MsgTitle) + End Try + End Sub +#End Region +#Region "Range" + Private DataPopulated As Boolean = False + Private Structure TPCELL + Private ReadOnly RowsCount As Integer + Private ReadOnly ColumnsCount As Integer + Friend ReadOnly Row As Integer + Friend ReadOnly Column As Integer + Friend Sub New(ByVal RowsCount As Integer, ByVal ColumnsCount As Integer) + Me.RowsCount = RowsCount - 1 + Me.ColumnsCount = ColumnsCount - 1 + Row = 0 + Column = 0 + End Sub + Private Sub New(ByVal RowsCount As Integer, ByVal ColumnsCount As Integer, ByVal Row As Integer, ByVal Column As Integer) + Me.New(RowsCount, ColumnsCount) + Me.Row = Row + Me.Column = Column + End Sub + Friend Function [Next]() As TPCELL + Dim r% = Row + Dim c% = Column + 1 + If Not c.ValueBetween(0, ColumnsCount) Then c = 0 : r += 1 + Return New TPCELL(RowsCount, ColumnsCount, r, c) + End Function + End Structure + Private Sub MyRange_IndexChanged(ByVal Sender As IRangeSwitcherProvider, ByVal e As EventArgs) Handles MyRange.IndexChanged + Try + If Sender.CurrentIndex >= 0 Then + AllowTopScroll = False + ScrollSuspended = True + Dim d As List(Of Integer) = MyRange.Indexes(Sender.CurrentIndex, EDP.ReturnValue).ListIfNothing + Dim i% + If d.Count > 0 Then + ClearTable() + If Sender.CurrentIndex > 0 And FeedEndless Then + i = MyRange.Indexes(Sender.CurrentIndex - 1, EDP.ReturnValue).DefaultIfEmpty(-1).Last + If i.ValueBetween(0, DataList.Count - 1) Then + If d.Count = 0 Then d.Add(i) Else d.Insert(0, i) + End If + End If + Dim w% = GetWidth() + Dim hp% = PaddingE.GetOf({TP_DATA}).Vertical(2) + Dim p As New TPCELL(DataRows, DataColumns) + Dim fmList As New List(Of FeedMedia) + Dim rhd As New Dictionary(Of Integer, List(Of Integer)) + For Each i In d + If i.ValueBetween(0, DataList.Count - 1) Then fmList.Add(New FeedMedia(DataList(i), w)) + Next + If fmList.Count > 0 Then fmList.ListDisposeRemoveAll(Function(fm) fm Is Nothing OrElse fm.HasError) + If fmList.Count > 0 Then + For i = 0 To fmList.Count - 1 + If Not rhd.ContainsKey(p.Row) Then rhd.Add(p.Row, New List(Of Integer)) + rhd(p.Row).Add(fmList(i).Height) + p = p.Next + Next + p = New TPCELL(DataRows, DataColumns) + ControlInvoke(TP_DATA, Sub() + With TP_DATA + With .RowStyles + For i = 0 To .Count - 1 + With .Item(i) : .SizeType = SizeType.Absolute : .Height = 0 : End With + Next + End With + .AutoScroll = False + .AutoScroll = True + End With + End Sub) + For i = 0 To fmList.Count - 1 + ControlInvoke(TP_DATA, Sub() + With TP_DATA + With .RowStyles(p.Row) : .SizeType = SizeType.Absolute : .Height = rhd(p.Row).Max : End With + .Controls.Add(fmList(i), p.Column, p.Row) + End With + End Sub) + p = p.Next + Next + End If + fmList.Clear() + rhd.ListForEach(Sub(kv, ii) kv.Value.Clear()) + rhd.Clear() + d.Clear() + End If + End If + Catch ex As Exception + ErrorsDescriber.Execute(EDP.SendInLog, ex, $"[DownloadObjects.DownloadFeedForm.Range.IndexChanged({Sender.CurrentIndex})]") + Finally + ControlInvoke(TP_DATA, Sub() + With TP_DATA.VerticalScroll + If Offset = 1 Then .Value = 0 Else .Value = .Maximum + End With + End Sub) + ScrollSuspended = False + DataPopulated = True + End Try + End Sub +#End Region +#Region "Size" + Private Function GetWidth() As Integer + Return (TP_DATA.Width - PaddingE.GetOf({Me, TP_DATA}).Horizontal(2)) / DataColumns + End Function + Private Sub DownloadFeedForm_ResizeEnd(sender As Object, e As EventArgs) Handles Me.ResizeEnd + ControlInvoke(TP_DATA, Sub() + With TP_DATA + If .Controls.Count > 0 Then + Dim w% = GetWidth() + Dim p As TableLayoutPanelCellPosition + Dim rh As New Dictionary(Of Integer, List(Of Integer)) + For Each cnt As FeedMedia In .Controls + cnt.Width = w + p = .GetCellPosition(cnt) + If Not rh.ContainsKey(p.Row) Then rh.Add(p.Row, New List(Of Integer)) + rh(p.Row).Add(cnt.Height) + Next + If rh.Count > 0 Then + For Each kv In rh + With .RowStyles(kv.Key) : .SizeType = SizeType.Absolute : .Height = kv.Value.Max : End With + kv.Value.Clear() + Next + End If + rh.Clear() + .AutoScroll = False + .AutoScroll = True + End If + End With + End Sub) + End Sub +#End Region +#Region "Scroll" + Private AllowTopScroll As Boolean = False + Private ScrollSuspended As Boolean = False + Private Offset As Integer = 1 + Private Sub TP_DATA_Paint(sender As Object, e As PaintEventArgs) Handles TP_DATA.Paint + If Not MyDefs.Initializing And Not ScrollSuspended And FeedEndless Then + ControlInvoke(TP_DATA, Sub() + With TP_DATA + Offset = IIf(.VerticalScroll.Value = 0 And AllowTopScroll, -1, 1) + If .VerticalScroll.Value + .VerticalScroll.LargeChange >= .DisplayRectangle.Height Or (.VerticalScroll.Value = 0 And AllowTopScroll) Then + If MyRange.TryMove(Offset) Then MyRange.Move(Offset) + End If + If Not AllowTopScroll And .VerticalScroll.Value > 0 Then AllowTopScroll = True + End With + End Sub) + End If + End Sub +#End Region + Private Sub ClearTable() + ControlInvoke(TP_DATA, Sub() + If TP_DATA.Controls.Count > 0 Then + For Each cnt As Control In TP_DATA.Controls : cnt.Dispose() : Next + TP_DATA.Controls.Clear() + End If + End Sub) + End Sub + End Class +End Namespace \ No newline at end of file diff --git a/SCrawler/Download/DownloadProgress.vb b/SCrawler/Download/DownloadProgress.vb index 9692176..ff087e4 100644 --- a/SCrawler/Download/DownloadProgress.vb +++ b/SCrawler/Download/DownloadProgress.vb @@ -11,8 +11,8 @@ Imports Download = SCrawler.Plugin.ISiteSettings.Download Imports TDJob = SCrawler.DownloadObjects.TDownloader.Job Namespace DownloadObjects Friend Class DownloadProgress : Implements IDisposable - Friend Event OnDownloadDone(ByVal Message As String) - Friend Event OnTotalCountChange() + Friend Event DownloadDone As NotificationEventHandler + Friend Event ProgressMaximumChanged() Private ReadOnly TP_MAIN As TableLayoutPanel Private ReadOnly TP_CONTROLS As TableLayoutPanel Private WithEvents BTT_START As Button @@ -89,10 +89,10 @@ Namespace DownloadObjects End If With Job - .Progress = New MyProgress(PR_MAIN, LBL_INFO) With {.DropCurrentProgressOnTotalChange = False} + .Progress = New MyProgress(PR_MAIN, LBL_INFO) With {.ResetProgressOnMaximumChanges = False} With .Progress - AddHandler .OnProgressChange, AddressOf JobProgress_OnProgressChange - AddHandler .OnTotalCountChange, AddressOf JobProgress_OnTotalCountChange + AddHandler .ProgressChanged, AddressOf JobProgress_ProgressChanged + AddHandler .MaximumChanged, AddressOf JobProgress_MaximumChanged End With End With @@ -139,7 +139,7 @@ Namespace DownloadObjects Job.Progress.InformationTemporary = $"{Job.Host.Name} downloading started" Job.Start() Instance.Download(Job.Token) - RaiseEvent OnDownloadDone($"Downloading saved {Job.Host.Name} posts is completed") + RaiseEvent DownloadDone($"Downloading saved {Job.Host.Name} posts is completed") Catch ex As Exception Job.Progress.InformationTemporary = $"{Job.Host.Name} downloading error" ErrorsDescriber.Execute(EDP.LogMessageValue, ex, {$"{Job.Host.Name} saved posts downloading error", "Saved posts"}) @@ -147,15 +147,15 @@ Namespace DownloadObjects btte.Invoke(BTT_START, True) btte.Invoke(BTT_STOP, False) Job.Stopped() - If Job.Type = Download.SavedPosts Then Job.Progress.TotalCount = 0 : Job.Progress.CurrentCounter = 0 + If Job.Type = Download.SavedPosts Then Job.Progress.Maximum = 0 : Job.Progress.Value = 0 End Try End Sub #End Region #Region "Progress, Jobs count" - Private Sub JobProgress_OnTotalCountChange(ByVal Source As IMyProgress, ByVal Index As Integer) - RaiseEvent OnTotalCountChange() + Private Sub JobProgress_MaximumChanged(ByVal Sender As Object, ByVal e As ProgressEventArgs) + RaiseEvent ProgressMaximumChanged() End Sub - Private Sub JobProgress_OnProgressChange(ByVal Source As IMyProgress, ByVal Index As Integer) + Private Sub JobProgress_ProgressChanged(ByVal Sender As Object, ByVal e As ProgressEventArgs) If Not Job.Type = Download.SavedPosts Then MainProgress.Perform() End Sub #End Region diff --git a/SCrawler/Download/DownloadSavedPostsForm.vb b/SCrawler/Download/DownloadSavedPostsForm.vb index b21d7e5..658d996 100644 --- a/SCrawler/Download/DownloadSavedPostsForm.vb +++ b/SCrawler/Download/DownloadSavedPostsForm.vb @@ -11,7 +11,7 @@ Imports PersonalUtilities.Forms Imports SCrawler.DownloadObjects Imports SCrawler.Plugin.Hosts Friend Class DownloadSavedPostsForm - Friend Event OnDownloadDone As NotificationEventHandler + Friend Event DownloadDone As NotificationEventHandler Private MyView As FormsView Private ReadOnly JobsList As List(Of DownloadProgress) Friend ReadOnly Property Working As Boolean @@ -45,7 +45,7 @@ Friend Class DownloadSavedPostsForm MyView.SetMeSize() If JobsList.Count > 0 Then For Each j As DownloadProgress In JobsList - AddHandler j.OnDownloadDone, AddressOf Jobs_OnDownloadDone + AddHandler j.DownloadDone, AddressOf Jobs_DownloadDone TP_MAIN.RowStyles.Add(New RowStyle(SizeType.Absolute, 60)) TP_MAIN.RowCount += 1 TP_MAIN.Controls.Add(j.Get, 0, TP_MAIN.RowStyles.Count - 1) @@ -65,8 +65,8 @@ Friend Class DownloadSavedPostsForm [Stop]() MyView.Dispose(Settings.Design) End Sub - Private Sub Jobs_OnDownloadDone(ByVal Message As String) - RaiseEvent OnDownloadDone(Message) + Private Sub Jobs_DownloadDone(ByVal Message As String) + RaiseEvent DownloadDone(Message) End Sub Private Sub BTT_DOWN_ALL_Click(sender As Object, e As EventArgs) Handles BTT_DOWN_ALL.Click Start() diff --git a/SCrawler/Download/DownloadedInfoForm.vb b/SCrawler/Download/DownloadedInfoForm.vb index 32021c6..8a969e6 100644 --- a/SCrawler/Download/DownloadedInfoForm.vb +++ b/SCrawler/Download/DownloadedInfoForm.vb @@ -96,7 +96,7 @@ Namespace DownloadObjects _TempUsersList.ListAddList(Downloader.Downloaded, LParams) Else _TempUsersList.ListAddList(Settings.Users.SelectMany(Of IUserData) _ - (Function(u) If(u.IsCollection, DirectCast(u, API.UserDataBind).Collections, {u})), LParams) + (Function(u) If(u.IsCollection, DirectCast(u, API.UserDataBind).Collections, {u})), LParams) End If If _TempUsersList.Count > 0 Then _TempUsersList.Sort(New UsersDateOrder) @@ -166,7 +166,7 @@ Namespace DownloadObjects Catch End Try End Sub - Friend Sub Downloader_OnDownloadCountChange() + Friend Sub Downloader_DownloadCountChange() If ViewMode = ViewModes.Session Then RefillList() End Sub Private Sub BTT_UP_Click(sender As Object, e As EventArgs) Handles BTT_UP.Click diff --git a/SCrawler/Download/FeedMedia.Designer.vb b/SCrawler/Download/FeedMedia.Designer.vb new file mode 100644 index 0000000..3202aeb --- /dev/null +++ b/SCrawler/Download/FeedMedia.Designer.vb @@ -0,0 +1,224 @@ +' Copyright (C) 2022 Andy +' This program is free software: you can redistribute it and/or modify +' it under the terms of the GNU General Public License as published by +' the Free Software Foundation, either version 3 of the License, or +' (at your option) any later version. +' +' This program is distributed in the hope that it will be useful, +' but WITHOUT ANY WARRANTY +Namespace DownloadObjects + + Partial Class FeedMedia : Inherits System.Windows.Forms.UserControl + + Protected Overrides Sub Dispose(ByVal disposing As Boolean) + Try + If disposing AndAlso components IsNot Nothing Then + components.Dispose() + End If + Finally + MyBase.Dispose(disposing) + End Try + End Sub + Private components As System.ComponentModel.IContainer + + Private Sub InitializeComponent() + Me.components = New System.ComponentModel.Container() + Dim CONTEXT_SEP_1 As System.Windows.Forms.ToolStripSeparator + Dim CONTEXT_SEP_2 As System.Windows.Forms.ToolStripSeparator + Dim CONTEXT_SEP_3 As System.Windows.Forms.ToolStripSeparator + Dim TP_LBL As System.Windows.Forms.TableLayoutPanel + Dim resources As System.ComponentModel.ComponentResourceManager = New System.ComponentModel.ComponentResourceManager(GetType(FeedMedia)) + Me.CH_CHECKED = New System.Windows.Forms.CheckBox() + Me.LBL_INFO = New System.Windows.Forms.Label() + Me.CONTEXT_DATA = New System.Windows.Forms.ContextMenuStrip(Me.components) + Me.BTT_CONTEXT_OPEN_MEDIA = New System.Windows.Forms.ToolStripMenuItem() + Me.BTT_CONTEXT_OPEN_USER = New System.Windows.Forms.ToolStripMenuItem() + Me.BTT_CONTEXT_OPEN_USER_URL = New System.Windows.Forms.ToolStripMenuItem() + Me.BTT_CONTEXT_OPEN_USER_POST = New System.Windows.Forms.ToolStripMenuItem() + Me.BTT_CONTEXT_FIND_USER = New System.Windows.Forms.ToolStripMenuItem() + Me.BTT_CONTEXT_DELETE = New System.Windows.Forms.ToolStripMenuItem() + Me.TP_MAIN = New System.Windows.Forms.TableLayoutPanel() + Me.MyVideo = New AxWMPLib.AxWindowsMediaPlayer() + Me.TT_MAIN = New System.Windows.Forms.ToolTip(Me.components) + Me.BTT_CONTEXT_INFO = New System.Windows.Forms.ToolStripMenuItem() + CONTEXT_SEP_1 = New System.Windows.Forms.ToolStripSeparator() + CONTEXT_SEP_2 = New System.Windows.Forms.ToolStripSeparator() + CONTEXT_SEP_3 = New System.Windows.Forms.ToolStripSeparator() + TP_LBL = New System.Windows.Forms.TableLayoutPanel() + TP_LBL.SuspendLayout() + Me.CONTEXT_DATA.SuspendLayout() + Me.TP_MAIN.SuspendLayout() + CType(Me.MyVideo, System.ComponentModel.ISupportInitialize).BeginInit() + Me.SuspendLayout() + ' + 'CONTEXT_SEP_1 + ' + CONTEXT_SEP_1.Name = "CONTEXT_SEP_1" + CONTEXT_SEP_1.Size = New System.Drawing.Size(177, 6) + ' + 'CONTEXT_SEP_2 + ' + CONTEXT_SEP_2.Name = "CONTEXT_SEP_2" + CONTEXT_SEP_2.Size = New System.Drawing.Size(177, 6) + ' + 'CONTEXT_SEP_3 + ' + CONTEXT_SEP_3.Name = "CONTEXT_SEP_3" + CONTEXT_SEP_3.Size = New System.Drawing.Size(177, 6) + ' + 'TP_LBL + ' + TP_LBL.ColumnCount = 2 + TP_LBL.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Absolute, 25.0!)) + TP_LBL.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Percent, 100.0!)) + TP_LBL.Controls.Add(Me.CH_CHECKED, 0, 0) + TP_LBL.Controls.Add(Me.LBL_INFO, 1, 0) + TP_LBL.Dock = System.Windows.Forms.DockStyle.Fill + TP_LBL.Location = New System.Drawing.Point(0, 0) + TP_LBL.Margin = New System.Windows.Forms.Padding(0) + TP_LBL.Name = "TP_LBL" + TP_LBL.RowCount = 1 + TP_LBL.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Percent, 100.0!)) + TP_LBL.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!)) + TP_LBL.Size = New System.Drawing.Size(146, 25) + TP_LBL.TabIndex = 0 + ' + 'CH_CHECKED + ' + Me.CH_CHECKED.AutoSize = True + Me.CH_CHECKED.CheckAlign = System.Drawing.ContentAlignment.MiddleCenter + Me.CH_CHECKED.Dock = System.Windows.Forms.DockStyle.Fill + Me.CH_CHECKED.Location = New System.Drawing.Point(3, 3) + Me.CH_CHECKED.Name = "CH_CHECKED" + Me.CH_CHECKED.Size = New System.Drawing.Size(19, 19) + Me.CH_CHECKED.TabIndex = 0 + Me.CH_CHECKED.UseVisualStyleBackColor = True + ' + 'LBL_INFO + ' + Me.LBL_INFO.AutoSize = True + Me.LBL_INFO.ContextMenuStrip = Me.CONTEXT_DATA + Me.LBL_INFO.Dock = System.Windows.Forms.DockStyle.Fill + Me.LBL_INFO.Location = New System.Drawing.Point(28, 0) + Me.LBL_INFO.Name = "LBL_INFO" + Me.LBL_INFO.Size = New System.Drawing.Size(115, 25) + Me.LBL_INFO.TabIndex = 1 + Me.LBL_INFO.TextAlign = System.Drawing.ContentAlignment.MiddleLeft + ' + 'CONTEXT_DATA + ' + Me.CONTEXT_DATA.Items.AddRange(New System.Windows.Forms.ToolStripItem() {Me.BTT_CONTEXT_OPEN_MEDIA, Me.BTT_CONTEXT_OPEN_USER, CONTEXT_SEP_1, Me.BTT_CONTEXT_OPEN_USER_URL, Me.BTT_CONTEXT_OPEN_USER_POST, CONTEXT_SEP_2, Me.BTT_CONTEXT_FIND_USER, Me.BTT_CONTEXT_INFO, CONTEXT_SEP_3, Me.BTT_CONTEXT_DELETE}) + Me.CONTEXT_DATA.Name = "CONTEXT_PIC" + Me.CONTEXT_DATA.Size = New System.Drawing.Size(181, 198) + ' + 'BTT_CONTEXT_OPEN_MEDIA + ' + Me.BTT_CONTEXT_OPEN_MEDIA.Image = Global.SCrawler.My.Resources.Resources.Folder_32 + Me.BTT_CONTEXT_OPEN_MEDIA.Name = "BTT_CONTEXT_OPEN_MEDIA" + Me.BTT_CONTEXT_OPEN_MEDIA.Size = New System.Drawing.Size(180, 22) + Me.BTT_CONTEXT_OPEN_MEDIA.Text = "Open" + ' + 'BTT_CONTEXT_OPEN_USER + ' + Me.BTT_CONTEXT_OPEN_USER.Image = Global.SCrawler.My.Resources.Resources.Folder_32 + Me.BTT_CONTEXT_OPEN_USER.Name = "BTT_CONTEXT_OPEN_USER" + Me.BTT_CONTEXT_OPEN_USER.Size = New System.Drawing.Size(180, 22) + Me.BTT_CONTEXT_OPEN_USER.Text = "Open user" + ' + 'BTT_CONTEXT_OPEN_USER_URL + ' + Me.BTT_CONTEXT_OPEN_USER_URL.Image = Global.SCrawler.My.Resources.Resources.GlobeBlue_32 + Me.BTT_CONTEXT_OPEN_USER_URL.Name = "BTT_CONTEXT_OPEN_USER_URL" + Me.BTT_CONTEXT_OPEN_USER_URL.Size = New System.Drawing.Size(180, 22) + Me.BTT_CONTEXT_OPEN_USER_URL.Text = "Open user" + ' + 'BTT_CONTEXT_OPEN_USER_POST + ' + Me.BTT_CONTEXT_OPEN_USER_POST.Image = Global.SCrawler.My.Resources.Resources.GlobeBlue_32 + Me.BTT_CONTEXT_OPEN_USER_POST.Name = "BTT_CONTEXT_OPEN_USER_POST" + Me.BTT_CONTEXT_OPEN_USER_POST.Size = New System.Drawing.Size(180, 22) + Me.BTT_CONTEXT_OPEN_USER_POST.Text = "Open post" + ' + 'BTT_CONTEXT_FIND_USER + ' + Me.BTT_CONTEXT_FIND_USER.Image = Global.SCrawler.My.Resources.Resources.InfoPic_32 + Me.BTT_CONTEXT_FIND_USER.Name = "BTT_CONTEXT_FIND_USER" + Me.BTT_CONTEXT_FIND_USER.Size = New System.Drawing.Size(180, 22) + Me.BTT_CONTEXT_FIND_USER.Text = "Find user" + ' + 'BTT_CONTEXT_DELETE + ' + Me.BTT_CONTEXT_DELETE.Image = Global.SCrawler.My.Resources.Resources.Delete + Me.BTT_CONTEXT_DELETE.Name = "BTT_CONTEXT_DELETE" + Me.BTT_CONTEXT_DELETE.Size = New System.Drawing.Size(180, 22) + Me.BTT_CONTEXT_DELETE.Text = "Delete" + ' + 'TP_MAIN + ' + Me.TP_MAIN.ColumnCount = 1 + Me.TP_MAIN.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Percent, 100.0!)) + Me.TP_MAIN.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Absolute, 20.0!)) + Me.TP_MAIN.Controls.Add(Me.MyVideo, 0, 1) + Me.TP_MAIN.Controls.Add(TP_LBL, 0, 0) + Me.TP_MAIN.Dock = System.Windows.Forms.DockStyle.Fill + Me.TP_MAIN.Location = New System.Drawing.Point(0, 0) + Me.TP_MAIN.Margin = New System.Windows.Forms.Padding(0) + Me.TP_MAIN.Name = "TP_MAIN" + Me.TP_MAIN.RowCount = 2 + Me.TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!)) + Me.TP_MAIN.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Percent, 100.0!)) + Me.TP_MAIN.Size = New System.Drawing.Size(146, 146) + Me.TP_MAIN.TabIndex = 0 + ' + 'MyVideo + ' + Me.MyVideo.Dock = System.Windows.Forms.DockStyle.Fill + Me.MyVideo.Enabled = True + Me.MyVideo.Location = New System.Drawing.Point(3, 28) + Me.MyVideo.Name = "MyVideo" + Me.MyVideo.OcxState = CType(resources.GetObject("MyVideo.OcxState"), System.Windows.Forms.AxHost.State) + Me.MyVideo.Size = New System.Drawing.Size(140, 115) + Me.MyVideo.TabIndex = 1 + ' + 'BTT_CONTEXT_INFO + ' + Me.BTT_CONTEXT_INFO.Image = Global.SCrawler.My.Resources.Resources.InfoPic_32 + Me.BTT_CONTEXT_INFO.Name = "BTT_CONTEXT_INFO" + Me.BTT_CONTEXT_INFO.Size = New System.Drawing.Size(180, 22) + Me.BTT_CONTEXT_INFO.Text = "Information" + ' + 'FeedMedia + ' + Me.AutoScaleDimensions = New System.Drawing.SizeF(6.0!, 13.0!) + Me.AutoScaleMode = System.Windows.Forms.AutoScaleMode.Font + Me.BackColor = System.Drawing.SystemColors.Window + Me.BorderStyle = System.Windows.Forms.BorderStyle.Fixed3D + Me.Controls.Add(Me.TP_MAIN) + Me.ForeColor = System.Drawing.SystemColors.WindowText + Me.Margin = New System.Windows.Forms.Padding(0) + Me.Name = "FeedMedia" + Me.Size = New System.Drawing.Size(146, 146) + TP_LBL.ResumeLayout(False) + TP_LBL.PerformLayout() + Me.CONTEXT_DATA.ResumeLayout(False) + Me.TP_MAIN.ResumeLayout(False) + CType(Me.MyVideo, System.ComponentModel.ISupportInitialize).EndInit() + Me.ResumeLayout(False) + + End Sub + + Private WithEvents TP_MAIN As TableLayoutPanel + Private WithEvents CONTEXT_DATA As ContextMenuStrip + Private WithEvents BTT_CONTEXT_OPEN_MEDIA As ToolStripMenuItem + Private WithEvents BTT_CONTEXT_OPEN_USER_URL As ToolStripMenuItem + Private WithEvents BTT_CONTEXT_OPEN_USER_POST As ToolStripMenuItem + Private WithEvents BTT_CONTEXT_FIND_USER As ToolStripMenuItem + Private WithEvents BTT_CONTEXT_DELETE As ToolStripMenuItem + Private WithEvents MyVideo As AxWMPLib.AxWindowsMediaPlayer + Private WithEvents BTT_CONTEXT_OPEN_USER As ToolStripMenuItem + Private WithEvents CH_CHECKED As CheckBox + Private WithEvents LBL_INFO As Label + Private WithEvents TT_MAIN As ToolTip + Private WithEvents BTT_CONTEXT_INFO As ToolStripMenuItem + End Class +End Namespace \ No newline at end of file diff --git a/SCrawler/Download/FeedMedia.resx b/SCrawler/Download/FeedMedia.resx new file mode 100644 index 0000000..aa5ae1b --- /dev/null +++ b/SCrawler/Download/FeedMedia.resx @@ -0,0 +1,149 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + text/microsoft-resx + + + 2.0 + + + System.Resources.ResXResourceReader, System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 + + + System.Resources.ResXResourceWriter, System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 + + + False + + + False + + + False + + + False + + + 17, 17 + + + + AAEAAAD/////AQAAAAAAAAAMAgAAAFdTeXN0ZW0uV2luZG93cy5Gb3JtcywgVmVyc2lvbj00LjAuMC4w + LCBDdWx0dXJlPW5ldXRyYWwsIFB1YmxpY0tleVRva2VuPWI3N2E1YzU2MTkzNGUwODkFAQAAACFTeXN0 + ZW0uV2luZG93cy5Gb3Jtcy5BeEhvc3QrU3RhdGUBAAAABERhdGEHAgIAAAAJAwAAAA8DAAAAtwAAAAIB + AAAAAQAAAAAAAAAAAAAAAKIAAAAAAwAACAAAAAAABQAAAAAAAADwPwMAAAAAAAUAAAAAAAAAAAAIAAIA + AAAAAAMAAQAAAAsA//8DAAAAAAALAP//CAACAAAAAAADADIAAAALAAAACAAKAAAAZgB1AGwAbAAAAAsA + AAALAAAACwD//wsA//8LAAAACAACAAAAAAAIAAIAAAAAAAgAAgAAAAAACAACAAAAAAALAAAA4g4AAE0M + AAAL + + + + 157, 17 + + \ No newline at end of file diff --git a/SCrawler/Download/FeedMedia.vb b/SCrawler/Download/FeedMedia.vb new file mode 100644 index 0000000..cf0ae35 --- /dev/null +++ b/SCrawler/Download/FeedMedia.vb @@ -0,0 +1,254 @@ +' Copyright (C) 2022 Andy +' This program is free software: you can redistribute it and/or modify +' it under the terms of the GNU General Public License as published by +' the Free Software Foundation, either version 3 of the License, or +' (at your option) any later version. +' +' This program is distributed in the hope that it will be useful, +' but WITHOUT ANY WARRANTY +'Imports System.Windows.Forms.PropertyGridInternal +Imports System.ComponentModel +Imports PersonalUtilities.Forms +Imports PersonalUtilities.Tools +Imports SCrawler.API.Base +Imports AxWMPLib +Imports UserMediaD = SCrawler.DownloadObjects.TDownloader.UserMediaD +Namespace DownloadObjects + + Public Class FeedMedia +#Region "Declarations" + Private Const VideoHeight As Integer = 450 + Private WithEvents MyPicture As PictureBox + Private ReadOnly MyImage As ImageRenderer + Friend ReadOnly Property Exists As Boolean + Get + Return Not MyPicture Is Nothing Or Not MyVideo Is Nothing + End Get + End Property + Friend ReadOnly Property HasError As Boolean + Friend ReadOnly File As SFile + Public Shadows Property Width As Integer + Get + Return MyBase.Width + End Get + Set(ByVal w As Integer) + If Size.Width <> w Then + Dim s As New Size(w, If(MyImage Is Nothing, VideoHeight, MyImage.FitToWidthF(w).Height)) + Dim objSize As Size = s + objSize.Height += (TP_MAIN.RowStyles(0).Height + PaddingE.GetOf({TP_MAIN}).Vertical(2)) + MinimumSize = objSize + MyBase.MaximumSize = objSize + Size = objSize + If Not MyImage Is Nothing Then + With MyPicture + .MinimumSize = Nothing + .MaximumSize = Nothing + .Size = s + .MinimumSize = s + .MaximumSize = s + End With + End If + End If + End Set + End Property + Private ReadOnly UserKey As String + Private ReadOnly Post As UserMedia + Friend ReadOnly Property Checked As Boolean + Get + Return CH_CHECKED.Checked + End Get + End Property + Friend ReadOnly Property Information As String +#End Region +#Region "Initializers" + Public Sub New() + InitializeComponent() + End Sub + Friend Sub New(ByVal Media As UserMediaD, ByVal Width As Integer) + Try + InitializeComponent() + File = Media.Data.File + If Not File.Exists And Media.Data.Type = UserMedia.Types.Video Then File.Path = $"{File.Path.CSFilePS}Video" + If Not File.Exists Then + If Not Media.Data.SpecialFolder.IsEmptyString Then + File.Path = $"{File.Path.CSFilePS}{Media.Data.SpecialFolder}".CSFileP + If Not File.Exists And Media.Data.Type = UserMedia.Types.Video Then File.Path = $"{File.Path.CSFilePS}Video" + End If + End If + If File.Exists Then + Information = $"Type: {Media.Data.Type}" + Information.StringAppendLine($"File: {File.File}") + Information.StringAppendLine($"Address: {File}") + Information.StringAppendLine($"Downloaded: {Media.Date.ToStringDate(ADateTime.Formats.BaseDateTime)}") + If Media.Data.Post.Date.HasValue Then Information.StringAppendLine($"Posted: {Media.Data.Post.Date.Value.ToStringDate(ADateTime.Formats.BaseDateTime)}") + Dim info$ = $"[{Media.Data.Type}] - " + Dim h% + Dim s As Size + + Post = Media.Data + + Select Case Media.Data.Type + Case UserMedia.Types.Picture, UserMedia.Types.GIF + MyImage = New ImageRenderer(File) + s = MyImage.FitToWidthF(Width) + h = s.Height + MyPicture = New PictureBox With { + .SizeMode = PictureBoxSizeMode.Zoom, + .Image = MyImage, + .InitialImage = .Image, + .Dock = DockStyle.None, + .Anchor = AnchorStyles.Left + AnchorStyles.Top, + .Size = s, + .MinimumSize = s, + .MaximumSize = s, + .Tag = File, + .Margin = New Padding(0), + .Padding = New Padding(0), + .ContextMenuStrip = CONTEXT_DATA + } + TP_MAIN.Controls.Remove(MyVideo) + MyVideo.Dispose() + MyVideo = Nothing + TP_MAIN.Controls.Add(MyPicture, 0, 1) + BTT_CONTEXT_OPEN_MEDIA.Text &= " picture" + BTT_CONTEXT_DELETE.Text &= " picture" + Case UserMedia.Types.Video + MyVideo.Tag = File + TP_MAIN.Controls.Add(MyVideo, 0, 1) + MyVideo.URL = File.ToString + BTT_CONTEXT_OPEN_MEDIA.Text &= " video" + BTT_CONTEXT_DELETE.Text &= " video" + h = VideoHeight + Case Else : Throw New ArgumentNullException With {.HelpLink = 1} + End Select + + If Not Media.User Is Nothing Then + With Media.User + UserKey = .Key + Information &= vbNewLine.StringDup(2) + If .IncludedInCollection Then Information.StringAppendLine($"User collection: { .CollectionName}") + Information.StringAppendLine($"User site: { .Site}") + Information.StringAppendLine($"User name: {IIf(Not .FriendlyName.IsEmptyString And Not .IncludedInCollection, .FriendlyName, .Name)}") + If .Site = API.Instagram.InstagramSite Then BTT_CONTEXT_OPEN_USER_POST.Visible = False + If .IncludedInCollection Then info &= $"[{ .CollectionName}]: " + info &= $"{ .Site} - {IIf(Not .FriendlyName.IsEmptyString And Not .IncludedInCollection, .FriendlyName, .Name)}" + End With + End If + + LBL_INFO.Text = info + TT_MAIN.SetToolTip(LBL_INFO, Information) + s = New Size(Width, h + TP_MAIN.RowStyles(0).Height + PaddingE.GetOf({TP_MAIN}).Vertical(2)) + Size = s + MinimumSize = s + MaximumSize = s + Else + Throw New ArgumentNullException With {.HelpLink = 1} + End If + Catch aex As ArgumentNullException When aex.HelpLink = 1 + HasError = True + Catch tex As Threading.ThreadStateException + HasError = True + Catch ex As Exception + ErrorsDescriber.Execute(EDP.SendInLog, ex, $"[DownloadObjects.FeedMedia({File})]") + HasError = True + End Try + End Sub +#End Region +#Region "Dispose" + Private Sub FeedImage_Disposed(sender As Object, e As EventArgs) Handles Me.Disposed + If Not MyImage Is Nothing Then MyImage.Dispose() + If Not MyPicture Is Nothing Then MyPicture.Dispose() + GC.SuppressFinalize(Me) + GC.Collect() + GC.WaitForFullGCComplete() + End Sub +#End Region +#Region "LBL" + Private Sub LBL_INFO_DoubleClick(sender As Object, e As EventArgs) Handles LBL_INFO.DoubleClick + If Not UserKey.IsEmptyString Then + Dim u As IUserData = Settings.GetUser(UserKey) + If Not u Is Nothing Then u.OpenFolder() + End If + End Sub +#End Region +#Region "Picture / Video objects" + Private Sub MyPicture_DoubleClick(sender As Object, e As EventArgs) Handles MyPicture.DoubleClick + Try : Process.Start(File) : Catch : End Try + End Sub + Private VideoPaused As Boolean = False + Private Sub MyVideo_PlayStateChange(sender As Object, e As _WMPOCXEvents_PlayStateChangeEvent) Handles MyVideo.PlayStateChange + Try + If Not VideoPaused Then + With MyVideo + If .playState = WMPLib.WMPPlayState.wmppsPlaying Then + .Ctlcontrols.currentPosition = 1 + .Ctlcontrols.pause() + VideoPaused = True + End If + End With + End If + Catch + End Try + End Sub +#End Region +#Region "Context" + Private Sub BTT_CONTEXT_OPEN_MEDIA_Click(sender As Object, e As EventArgs) Handles BTT_CONTEXT_OPEN_MEDIA.Click + File.Open(, EDP.None) + End Sub + Private Sub BTT_CONTEXT_OPEN_USER_Click(sender As Object, e As EventArgs) Handles BTT_CONTEXT_OPEN_USER.Click + If Not UserKey.IsEmptyString Then + Dim u As IUserData = Settings.GetUser(UserKey) + If Not u Is Nothing Then u.OpenFolder() + End If + End Sub + Private Sub BTT_CONTEXT_OPEN_USER_URL_Click(sender As Object, e As EventArgs) Handles BTT_CONTEXT_OPEN_USER_URL.Click + If Not UserKey.IsEmptyString Then + Dim u As IUserData = Settings.GetUser(UserKey) + If Not u Is Nothing Then u.OpenSite() + End If + End Sub + Private Sub BTT_CONTEXT_OPEN_USER_POST_Click(sender As Object, e As EventArgs) Handles BTT_CONTEXT_OPEN_USER_POST.Click + Try + If Not UserKey.IsEmptyString And Not Post.Post.ID.IsEmptyString Then + Dim u As IUserData = Settings.GetUser(UserKey) + If Not u Is Nothing Then + Dim url$ = UserDataBase.GetPostUrl(u, Post) + If Not url.IsEmptyString Then + Try : Process.Start(url) : Catch : End Try + End If + End If + End If + Catch ex As Exception + ErrorsDescriber.Execute(EDP.LogMessageValue, ex, $"[FeedMedia.OpenPost({UserKey}, {Post.Post.ID})]") + End Try + End Sub + Private Sub BTT_CONTEXT_FIND_USER_Click(sender As Object, e As EventArgs) Handles BTT_CONTEXT_FIND_USER.Click + If Not UserKey.IsEmptyString Then MainFrameObj.FocusUser(UserKey, True) + End Sub + Private Sub BTT_CONTEXT_INFO_Click(sender As Object, e As EventArgs) Handles BTT_CONTEXT_INFO.Click + MsgBoxE({Information, "Post information"}) + End Sub + Private Sub BTT_CONTEXT_DELETE_Click(sender As Object, e As EventArgs) Handles BTT_CONTEXT_DELETE.Click + DeleteFile(False) + End Sub + Friend Function DeleteFile(ByVal Silent As Boolean) As Boolean + Const msgTitle$ = "Deleting a file" + Try + If Silent OrElse MsgBoxE({$"Are you sure you want to delete the [{File.File}] file?{vbCr}{File}", msgTitle}, vbExclamation,,, {"Process", "Cancel"}) = 0 Then + If File.Delete(SFO.File, Settings.DeleteMode, EDP.ThrowException) Then + If Not Silent Then MsgBoxE({"File deleted", msgTitle}) + LBL_INFO.Height = 0 + If Not MyPicture Is Nothing Then MyPicture.Size = New Size(0, 0) + Height = 0 + Return True + End If + End If + Return False + Catch ex As Exception + Dim e As New ErrorsDescriber(EDP.LogMessageValue) With {.ShowMainMsg = Not Silent, .ShowExMsg = .ShowMainMsg} + Return ErrorsDescriber.Execute(e, ex, $"[FeedMedia.DeleteFile({File})]", False) + End Try + End Function +#End Region + End Class +End Namespace \ No newline at end of file diff --git a/SCrawler/Download/Groups/DownloadGroup.vb b/SCrawler/Download/Groups/DownloadGroup.vb index fa176cd..1af65b7 100644 --- a/SCrawler/Download/Groups/DownloadGroup.vb +++ b/SCrawler/Download/Groups/DownloadGroup.vb @@ -48,16 +48,16 @@ Namespace DownloadObjects.Groups } BTT_DELETE = New ToolStripMenuItem With { .Image = PersonalUtilities.My.Resources.DeletePic_02_Red_24, - .BackColor = ColorBttDeleteBack, - .ForeColor = ColorBttDeleteFore, + .BackColor = MyColor.DeleteBack, + .ForeColor = MyColor.DeleteFore, .Text = "Delete", .ToolTipText = String.Empty, .AutoToolTip = False } BTT_EDIT = New ToolStripMenuItem With { .Image = PersonalUtilities.My.Resources.PencilPic_01_48, - .BackColor = ColorBttEditBack, - .ForeColor = ColorBttEditFore, + .BackColor = MyColor.EditBack, + .ForeColor = MyColor.EditFore, .Text = "Edit", .ToolTipText = String.Empty, .AutoToolTip = False diff --git a/SCrawler/Download/MissingPostsForm.Designer.vb b/SCrawler/Download/MissingPostsForm.Designer.vb new file mode 100644 index 0000000..ebf3b87 --- /dev/null +++ b/SCrawler/Download/MissingPostsForm.Designer.vb @@ -0,0 +1,175 @@ +' Copyright (C) 2022 Andy +' This program is free software: you can redistribute it and/or modify +' it under the terms of the GNU General Public License as published by +' the Free Software Foundation, either version 3 of the License, or +' (at your option) any later version. +' +' This program is distributed in the hope that it will be useful, +' but WITHOUT ANY WARRANTY +Namespace DownloadObjects + + Partial Friend Class MissingPostsForm : Inherits System.Windows.Forms.Form + + Protected Overrides Sub Dispose(ByVal disposing As Boolean) + Try + If disposing AndAlso components IsNot Nothing Then + components.Dispose() + End If + Finally + MyBase.Dispose(disposing) + End Try + End Sub + Private components As System.ComponentModel.IContainer + + Private Sub InitializeComponent() + Me.components = New System.ComponentModel.Container() + Dim CONTAINER_MAIN As System.Windows.Forms.ToolStripContainer + Dim CONTEXT_SEP_1 As System.Windows.Forms.ToolStripSeparator + Dim CONTEXT_SEP_2 As System.Windows.Forms.ToolStripSeparator + Dim CONTEXT_SEP_3 As System.Windows.Forms.ToolStripSeparator + Me.LIST_DATA = New System.Windows.Forms.ListView() + Me.CONTEXT_OPERATIONS = New System.Windows.Forms.ContextMenuStrip(Me.components) + Me.BTT_DOWN = New System.Windows.Forms.ToolStripMenuItem() + Me.BTT_OPEN_POST = New System.Windows.Forms.ToolStripMenuItem() + Me.BTT_OPEN_USER = New System.Windows.Forms.ToolStripMenuItem() + Me.BTT_FIND_USER = New System.Windows.Forms.ToolStripMenuItem() + Me.BTT_DELETE = New System.Windows.Forms.ToolStripMenuItem() + Me.BTT_CONTEXT_SHOW_POST_INFO = New System.Windows.Forms.ToolStripMenuItem() + CONTAINER_MAIN = New System.Windows.Forms.ToolStripContainer() + CONTEXT_SEP_1 = New System.Windows.Forms.ToolStripSeparator() + CONTEXT_SEP_2 = New System.Windows.Forms.ToolStripSeparator() + CONTEXT_SEP_3 = New System.Windows.Forms.ToolStripSeparator() + CONTAINER_MAIN.ContentPanel.SuspendLayout() + CONTAINER_MAIN.SuspendLayout() + Me.CONTEXT_OPERATIONS.SuspendLayout() + Me.SuspendLayout() + ' + 'CONTAINER_MAIN + ' + CONTAINER_MAIN.BottomToolStripPanelVisible = False + ' + 'CONTAINER_MAIN.ContentPanel + ' + CONTAINER_MAIN.ContentPanel.Controls.Add(Me.LIST_DATA) + CONTAINER_MAIN.ContentPanel.Size = New System.Drawing.Size(284, 236) + CONTAINER_MAIN.Dock = System.Windows.Forms.DockStyle.Fill + CONTAINER_MAIN.LeftToolStripPanelVisible = False + CONTAINER_MAIN.Location = New System.Drawing.Point(0, 0) + CONTAINER_MAIN.Name = "CONTAINER_MAIN" + CONTAINER_MAIN.RightToolStripPanelVisible = False + CONTAINER_MAIN.Size = New System.Drawing.Size(284, 261) + CONTAINER_MAIN.TabIndex = 0 + ' + 'LIST_DATA + ' + Me.LIST_DATA.Activation = System.Windows.Forms.ItemActivation.OneClick + Me.LIST_DATA.ContextMenuStrip = Me.CONTEXT_OPERATIONS + Me.LIST_DATA.Dock = System.Windows.Forms.DockStyle.Fill + Me.LIST_DATA.HeaderStyle = System.Windows.Forms.ColumnHeaderStyle.None + Me.LIST_DATA.HideSelection = False + Me.LIST_DATA.Location = New System.Drawing.Point(0, 0) + Me.LIST_DATA.Name = "LIST_DATA" + Me.LIST_DATA.Size = New System.Drawing.Size(284, 236) + Me.LIST_DATA.TabIndex = 0 + Me.LIST_DATA.TileSize = New System.Drawing.Size(168, 15) + Me.LIST_DATA.UseCompatibleStateImageBehavior = False + Me.LIST_DATA.View = System.Windows.Forms.View.Tile + ' + 'CONTEXT_SEP_1 + ' + CONTEXT_SEP_1.Name = "CONTEXT_SEP_1" + CONTEXT_SEP_1.Size = New System.Drawing.Size(178, 6) + ' + 'CONTEXT_SEP_2 + ' + CONTEXT_SEP_2.Name = "CONTEXT_SEP_2" + CONTEXT_SEP_2.Size = New System.Drawing.Size(178, 6) + ' + 'CONTEXT_SEP_3 + ' + CONTEXT_SEP_3.Name = "CONTEXT_SEP_3" + CONTEXT_SEP_3.Size = New System.Drawing.Size(178, 6) + ' + 'CONTEXT_OPERATIONS + ' + Me.CONTEXT_OPERATIONS.Items.AddRange(New System.Windows.Forms.ToolStripItem() {Me.BTT_DOWN, CONTEXT_SEP_1, Me.BTT_OPEN_POST, Me.BTT_OPEN_USER, Me.BTT_CONTEXT_SHOW_POST_INFO, CONTEXT_SEP_2, Me.BTT_FIND_USER, CONTEXT_SEP_3, Me.BTT_DELETE}) + Me.CONTEXT_OPERATIONS.Name = "CONTEXT_OPERATIONS" + Me.CONTEXT_OPERATIONS.Size = New System.Drawing.Size(181, 176) + ' + 'BTT_DOWN + ' + Me.BTT_DOWN.AutoToolTip = True + Me.BTT_DOWN.Image = Global.SCrawler.My.Resources.Resources.StartPic_01_Green_16 + Me.BTT_DOWN.Name = "BTT_DOWN" + Me.BTT_DOWN.Size = New System.Drawing.Size(181, 22) + Me.BTT_DOWN.Text = "Download" + Me.BTT_DOWN.ToolTipText = "Try downloading the selected posts again" + ' + 'BTT_OPEN_POST + ' + Me.BTT_OPEN_POST.Image = Global.SCrawler.My.Resources.Resources.GlobeBlue_32 + Me.BTT_OPEN_POST.Name = "BTT_OPEN_POST" + Me.BTT_OPEN_POST.Size = New System.Drawing.Size(181, 22) + Me.BTT_OPEN_POST.Text = "Open post" + ' + 'BTT_OPEN_USER + ' + Me.BTT_OPEN_USER.Image = Global.SCrawler.My.Resources.Resources.Folder_32 + Me.BTT_OPEN_USER.Name = "BTT_OPEN_USER" + Me.BTT_OPEN_USER.Size = New System.Drawing.Size(181, 22) + Me.BTT_OPEN_USER.Text = "Open user folder" + ' + 'BTT_FIND_USER + ' + Me.BTT_FIND_USER.Image = Global.SCrawler.My.Resources.Resources.InfoPic_32 + Me.BTT_FIND_USER.Name = "BTT_FIND_USER" + Me.BTT_FIND_USER.Size = New System.Drawing.Size(181, 22) + Me.BTT_FIND_USER.Text = "Find user" + ' + 'BTT_DELETE + ' + Me.BTT_DELETE.AutoToolTip = True + Me.BTT_DELETE.Image = Global.SCrawler.My.Resources.Resources.Delete + Me.BTT_DELETE.Name = "BTT_DELETE" + Me.BTT_DELETE.Size = New System.Drawing.Size(181, 22) + Me.BTT_DELETE.Text = "Delete post" + Me.BTT_DELETE.ToolTipText = "Remove selected posts from user data" + ' + 'BTT_CONTEXT_SHOW_POST_INFO + ' + Me.BTT_CONTEXT_SHOW_POST_INFO.AutoToolTip = True + Me.BTT_CONTEXT_SHOW_POST_INFO.Image = Global.SCrawler.My.Resources.Resources.InfoPic_32 + Me.BTT_CONTEXT_SHOW_POST_INFO.Name = "BTT_CONTEXT_SHOW_POST_INFO" + Me.BTT_CONTEXT_SHOW_POST_INFO.Size = New System.Drawing.Size(180, 22) + Me.BTT_CONTEXT_SHOW_POST_INFO.Text = "Show post info" + Me.BTT_CONTEXT_SHOW_POST_INFO.ToolTipText = "Show information about the missing post" + ' + 'MissingPostsForm + ' + Me.AutoScaleDimensions = New System.Drawing.SizeF(6.0!, 13.0!) + Me.AutoScaleMode = System.Windows.Forms.AutoScaleMode.Font + Me.ClientSize = New System.Drawing.Size(284, 261) + Me.Controls.Add(CONTAINER_MAIN) + Me.KeyPreview = True + Me.MinimumSize = New System.Drawing.Size(300, 300) + Me.Name = "MissingPostsForm" + Me.ShowIcon = False + Me.Text = "Missing posts" + CONTAINER_MAIN.ContentPanel.ResumeLayout(False) + CONTAINER_MAIN.ResumeLayout(False) + CONTAINER_MAIN.PerformLayout() + Me.CONTEXT_OPERATIONS.ResumeLayout(False) + Me.ResumeLayout(False) + + End Sub + + Private WithEvents LIST_DATA As ListView + Private WithEvents CONTEXT_OPERATIONS As ContextMenuStrip + Private WithEvents BTT_DOWN As ToolStripMenuItem + Private WithEvents BTT_OPEN_POST As ToolStripMenuItem + Private WithEvents BTT_OPEN_USER As ToolStripMenuItem + Private WithEvents BTT_FIND_USER As ToolStripMenuItem + Private WithEvents BTT_DELETE As ToolStripMenuItem + Private WithEvents BTT_CONTEXT_SHOW_POST_INFO As ToolStripMenuItem + End Class +End Namespace \ No newline at end of file diff --git a/SCrawler/Download/MissingPostsForm.resx b/SCrawler/Download/MissingPostsForm.resx new file mode 100644 index 0000000..f048d80 --- /dev/null +++ b/SCrawler/Download/MissingPostsForm.resx @@ -0,0 +1,135 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + text/microsoft-resx + + + 2.0 + + + System.Resources.ResXResourceReader, System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 + + + System.Resources.ResXResourceWriter, System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 + + + False + + + 17, 17 + + + False + + + False + + + False + + \ No newline at end of file diff --git a/SCrawler/Download/MissingPostsForm.vb b/SCrawler/Download/MissingPostsForm.vb new file mode 100644 index 0000000..3eeb106 --- /dev/null +++ b/SCrawler/Download/MissingPostsForm.vb @@ -0,0 +1,296 @@ +' Copyright (C) 2022 Andy +' This program is free software: you can redistribute it and/or modify +' it under the terms of the GNU General Public License as published by +' the Free Software Foundation, either version 3 of the License, or +' (at your option) any later version. +' +' This program is distributed in the hope that it will be useful, +' but WITHOUT ANY WARRANTY +Imports System.ComponentModel +Imports PersonalUtilities.Functions.Messaging +Imports PersonalUtilities.Forms +Imports PersonalUtilities.Forms.Toolbars +Imports SCrawler.API.Base +Namespace DownloadObjects + Friend Class MissingPostsForm +#Region "Declarations" + Private WithEvents MyDefs As DefaultFormOptions + Private ReadOnly MUsers As List(Of IUserData) + Private WithEvents BTT_DOWN_ALL As ToolStripButton + Private WithEvents BTT_INFO As ToolStripButton +#End Region +#Region "Initializer" + Friend Sub New() + InitializeComponent() + MUsers = New List(Of IUserData) + MyDefs = New DefaultFormOptions(Me, Settings.Design) + BTT_DOWN_ALL = New ToolStripButton With { + .Text = "Download ALL", + .ToolTipText = String.Empty, + .AutoToolTip = False, + .Image = My.Resources.StartPic_01_Green_16, + .DisplayStyle = ToolStripItemDisplayStyle.ImageAndText + } + BTT_INFO = New ToolStripButton With { + .Text = "Info", + .ToolTipText = "Show information about the missing post (F1)", + .AutoToolTip = True, + .Image = My.Resources.InfoPic_32, + .DisplayStyle = ToolStripItemDisplayStyle.ImageAndText + } + End Sub +#End Region +#Region "Form handlers" + Private Sub MissingPostsForm_Load(sender As Object, e As EventArgs) Handles Me.Load + With MyDefs + .MyViewInitialize() + .AddEditToolbarPlus({EditToolbar.ControlItem.Separator, BTT_DOWN_ALL, BTT_INFO}) + .EndLoaderOperations(False) + End With + RefillList() + End Sub + Private Sub MissingPostsForm_Closing(sender As Object, e As CancelEventArgs) Handles Me.Closing + e.Cancel = True + Hide() + End Sub + Private Sub MissingPostsForm_Disposed(sender As Object, e As EventArgs) Handles Me.Disposed + MUsers.Clear() + End Sub + Private Sub MissingPostsForm_KeyDown(sender As Object, e As KeyEventArgs) Handles Me.KeyDown + If e.KeyCode = Keys.F1 Then ShowPostInformation() : e.Handled = True + End Sub +#End Region +#Region "RefillList" + Private Overloads Sub RefillList() Handles MyDefs.ButtonUpdateClick + RefillList(True) + End Sub + Friend Overloads Sub RefillList(ByVal User As IUserData) + If MUsers.Count = 0 OrElse Not MUsers.Contains(User) Then MUsers.Add(User) : RefillList(False) + End Sub + Friend Overloads Sub RefillList(ByVal Reload As Boolean) + Try + If Reload Then MUsers.Clear() + LIST_DATA.Items.Clear() + LIST_DATA.Groups.Clear() + If Reload And Settings.Users.Count > 0 Then + MUsers.ListAddList(Settings.Users.SelectMany(Function(ByVal user As IUserData) As IEnumerable(Of IUserData) + DirectCast(user, UserDataBase).LoadContentInformation() + If user.IsCollection Then + With DirectCast(user, API.UserDataBind) + If .Count > 0 Then Return .Where(Function(u) DirectCast(u, UserDataBase).ContentMissingExists) + End With + ElseIf DirectCast(user, UserDataBase).ContentMissingExists Then + Return {user} + End If + Return New IUserData() {} + End Function), LAP.IgnoreICopier) + End If + If MUsers.Count > 0 Then + Dim gName$ = String.Empty + Dim g As ListViewGroup = Nothing + Dim i% = -1 + Dim cm As List(Of UserMedia) + For Each uu As UserDataBase In MUsers + i += 1 + cm = uu.ContentMissing + If cm.Count > 0 Then + gName = String.Empty + If uu.IncludedInCollection Then gName = $"{uu.CollectionName} - " + gName &= $"{uu.User.Name} ({uu.Site})" + ControlInvoke(LIST_DATA, Sub() + LIST_DATA.Groups.Add(New ListViewGroup(gName) With {.Tag = uu.LVIKey}) + g = LIST_DATA.Groups(LIST_DATA.Groups.Count - 1) + End Sub) + For i% = 0 To cm.Count - 1 : ControlInvoke(LIST_DATA, Sub() LIST_DATA.Items.Add(New ListViewItem(cm(i).Post.ID, g))) : Next + End If + Next + End If + Catch ex As Exception + ErrorsDescriber.Execute(EDP.SendInLog, ex, "[DownloadObjects.MissingPostsForm.RefillList]") + End Try + End Sub +#End Region +#Region "Post actions" + Private Sub BTT_DOWN_Click(sender As Object, e As EventArgs) Handles BTT_DOWN.Click + Try + If LIST_DATA.SelectedItems.Count > 0 Then + Dim users As List(Of IUserData) = LIST_DATA.SelectedItems.ToObjectsList.ListCast(Of ListViewItem)(). + Select(Function(d) Settings.GetUser(CStr(d.Group.Tag))).ListWithRemove(Function(d) d Is Nothing) + If users.ListExists Then + If MsgBoxE({"The following users will be added to the download queue:" & vbCr & vbCr & + users.Select(Function(u) u.ToString).ListToString(vbNewLine), "Download users"},,,, {"Process", "Cancel"}) = 0 Then + users.ForEach(Sub(u) u.DownloadMissingOnly = True) + Downloader.AddRange(users) + users.Clear() + End If + End If + Else + MsgBoxE("No selected posts") + End If + Catch ex As Exception + ErrorsDescriber.Execute(EDP.SendInLog, ex, "[DownloadObjects.MissingPostsForm.Download]") + End Try + End Sub + Private Sub BTT_OPEN_POST_Click(sender As Object, e As EventArgs) Handles BTT_OPEN_POST.Click + Try + If LIST_DATA.SelectedItems.Count > 0 Then + If LIST_DATA.SelectedItems.Count = 1 OrElse + MsgBoxE({$"Are you sure you want to open {LIST_DATA.SelectedItems.Count} posts?", "Open multiple posts"}, vbExclamation + vbYesNo) = vbYes Then + Dim data As List(Of ListViewItem) = LIST_DATA.SelectedItems.ToObjectsList.ListCast(Of ListViewItem) + If data.ListExists Then + Dim uKey$, url$ + Dim u As IUserData = Nothing + Dim i% + Dim cm As List(Of UserMedia) + For Each _d In data + uKey = _d.Group.Tag + If u Is Nothing OrElse Not u.Key = uKey Then u = Settings.GetUser(uKey) + If Not u Is Nothing Then + i = -1 + With DirectCast(u, UserDataBase) + cm = .ContentMissing + If cm.Count > 0 Then i = cm.FindIndex(Function(c) c.Post.ID = _d.Text) + If i >= 0 Then + url = UserDataBase.GetPostUrl(u, cm(i)) + If Not url.IsEmptyString Then + Try : Process.Start(url) : Catch : End Try + End If + End If + End With + End If + Next + End If + End If + End If + Catch ex As Exception + ErrorsDescriber.Execute(EDP.SendInLog, ex, "[DownloadObjects.MissingPostsForm.OpenPost]") + End Try + End Sub + Private Sub BTT_OPEN_USER_Click(sender As Object, e As EventArgs) Handles BTT_OPEN_USER.Click + Try + If LIST_DATA.SelectedItems.Count > 0 Then + Dim users As List(Of IUserData) = LIST_DATA.SelectedItems.ToObjectsList.ListCast(Of ListViewItem)(). + Select(Function(d) Settings.GetUser(CStr(d.Group.Tag))).ListWithRemove(Function(d) d Is Nothing) + If users.ListExists Then users.ForEach(Sub(u) u.OpenFolder()) + Else + MsgBoxE("No selected posts") + End If + Catch ex As Exception + ErrorsDescriber.Execute(EDP.SendInLog, ex, $"[DownloadObjects.MissingPostsForm.OpenUser]") + End Try + End Sub + Private Sub ShowPostInformation() Handles BTT_INFO.Click, BTT_CONTEXT_SHOW_POST_INFO.Click, LIST_DATA.DoubleClick + Try + If LIST_DATA.SelectedItems.Count > 0 Then + Dim data As ListViewItem = LIST_DATA.SelectedItems.ToObjectsList.ListCast(Of ListViewItem)().First + Dim uKey$, url$ + Dim u As IUserData = Nothing + Dim i% + Dim cm As List(Of UserMedia) + Dim m As UserMedia + uKey = data.Group.Tag + If Not uKey.IsEmptyString Then u = Settings.GetUser(uKey) + If Not u Is Nothing Then + i = -1 + With DirectCast(u, UserDataBase) + cm = .ContentMissing + If cm.Count > 0 Then i = cm.FindIndex(Function(c) c.Post.ID = data.Text) + If i >= 0 Then + m = cm(i) + url = UserDataBase.GetPostUrl(u, m) + Dim msg As New MMessage("", "Post information") With {.Editable = True} + Dim b As New List(Of MsgBoxButton) + If Not url.IsEmptyString Then b.Add(New MsgBoxButton("Open") With {.IsDialogResultButton = False, + .ToolTip = "Open post in browser", + .KeyCode = Keys.F1, + .CallBack = Sub(result, message, button) + Try : Process.Start(url) : Catch : End Try + End Sub}) + b.Add(New MsgBoxButton("OK")) + With msg + .Buttons = b + .DefaultButton = If(b.Count = 2, 1, 0) + .CancelButton = .DefaultButton + .Text = $"Type: {m.Type}" + .Text.StringAppendLine($"Address: {url}") + If m.Post.Date.HasValue Then .Text.StringAppendLine($"Date: {m.Post.Date.Value.ToStringDate(ADateTime.Formats.BaseDateTime)}") + .Text &= vbNewLine.StringDup(2) + If u.IncludedInCollection Then .Text.StringAppendLine($"User collection: {u.CollectionName}") + .Text.StringAppendLine($"User site: {u.Site}") + .Text.StringAppendLine($"User name: {IIf(Not u.FriendlyName.IsEmptyString And Not u.IncludedInCollection, u.FriendlyName, u.Name)}") + End With + MsgBoxE(msg) + b.Clear() + cm.Clear() + End If + End With + End If + Else + MsgBoxE("No selected posts") + End If + Catch ex As Exception + ErrorsDescriber.Execute(EDP.SendInLog, ex, "[DownloadObjects.MissingPostsForm.ShowPostInformation]") + End Try + End Sub + Private Sub BTT_FIND_USER_Click(sender As Object, e As EventArgs) Handles BTT_FIND_USER.Click + Try + If LIST_DATA.SelectedItems.Count > 0 Then + Dim user As IUserData = LIST_DATA.SelectedItems.ToObjectsList.ListCast(Of ListViewItem)(). + Select(Function(d) Settings.GetUser(CStr(d.Group.Tag))).ListWithRemove(Function(d) d Is Nothing).DefaultIfEmpty(Nothing).First + If Not user Is Nothing Then MainFrameObj.FocusUser(user.Key, True) + Else + MsgBoxE("No selected posts") + End If + Catch ex As Exception + ErrorsDescriber.Execute(EDP.SendInLog, ex, $"[DownloadObjects.MissingPostsForm.FindUser]") + End Try + End Sub + Private Sub DeletePost() Handles MyDefs.ButtonDeleteClickE, BTT_DELETE.Click + Const MsgTitle$ = "Remove missing posts" + Dim UsersToUpdate As New List(Of UserDataBase) + Try + Dim data As List(Of ListViewItem) = LIST_DATA.SelectedItems.ToObjectsList.ListCast(Of ListViewItem) + If data.ListExists Then + Dim lp As New ListAddParams(LAP.NotContainsOnly) + Dim usersCount% = ListAddList(Nothing, data.Select(Function(d) d.Group.Name), LAP.NotContainsOnly).ListIfNothing.Count + If MsgBoxE({"Are you sure you want to delete the selected missing posts?" & vbCr & + $"Number of affected users: {usersCount}." & vbCr & + $"Number of posts to be deleted: {data.Count}.", MsgTitle}, vbExclamation,,, {"Process", "Cancel"}) = 0 Then + Dim uKey$ + Dim u As UserDataBase = Nothing + Dim cm As List(Of UserMedia) + Dim i% + For Each _d In data + uKey = _d.Group.Tag + If u Is Nothing OrElse Not u.LVIKey = uKey Then u = Settings.GetUser(uKey) + If Not u Is Nothing Then + i = -1 + cm = u.ContentMissing + If cm.Count > 0 Then i = cm.FindIndex(Function(c) c.Post.ID = _d.Text) + If i >= 0 Then u.RemoveMedia(cm(i), UserMedia.States.Missing) : UsersToUpdate.ListAddValue(u, lp) + End If + Next + MsgBoxE({"The selected posts have been successfully deleted", MsgTitle}) + Else + MsgBoxE({"Operation canceled", MsgTitle}) + End If + Else + MsgBoxE({"No selected posts", MsgTitle}) + End If + Catch ex As Exception + ErrorsDescriber.Execute(EDP.SendInLog, ex, "[DownloadObjects.MissingPostsForm.DeletePost]") + Finally + UpdateUsers(UsersToUpdate) + UsersToUpdate.Clear() + End Try + End Sub + Private Sub UpdateUsers(ByVal UserList As List(Of UserDataBase)) + Try + If UserList.ListExists Then UserList.ForEach(Sub(u) u.UpdateContentInformation()) + Catch ex As Exception + ErrorsDescriber.Execute(EDP.SendInLog, ex, "[DownloadObjects.MissingPostsForm.UpdateUsers]") + End Try + End Sub +#End Region + End Class +End Namespace \ No newline at end of file diff --git a/SCrawler/Download/Scheduler.vb b/SCrawler/Download/Scheduler.vb index 0132acb..0577628 100644 --- a/SCrawler/Download/Scheduler.vb +++ b/SCrawler/Download/Scheduler.vb @@ -13,10 +13,6 @@ Imports System.Threading Namespace DownloadObjects Friend Class Scheduler : Implements IEnumerable(Of AutoDownloader), IMyEnumerator(Of AutoDownloader), IDisposable Friend Const Name_Plan As String = "Plan" - Friend Event UserFind As AutoDownloader.UserFindEventHandler - Private Sub OnUserFind(ByVal Key As String, ByVal Activate As Boolean) - RaiseEvent UserFind(Key, Activate) - End Sub Private ReadOnly Plans As List(Of AutoDownloader) Private ReadOnly File As SFile = $"Settings\AutoDownload.xml" Private ReadOnly PlanWorking As Predicate(Of AutoDownloader) = Function(Plan) Plan.Working @@ -36,10 +32,7 @@ Namespace DownloadObjects End If End Using End If - If Plans.Count > 0 Then Plans.ForEach(Sub(p) - p.Source = Me - AddHandler p.UserFind, AddressOf OnUserFind - End Sub) + If Plans.Count > 0 Then Plans.ForEach(Sub(p) p.Source = Me) End Sub Default Friend ReadOnly Property Item(ByVal Index As Integer) As AutoDownloader Implements IMyEnumerator(Of AutoDownloader).MyEnumeratorObject Get @@ -56,7 +49,6 @@ Namespace DownloadObjects End Function Friend Sub Add(ByVal Plan As AutoDownloader) Plan.Source = Me - AddHandler Plan.UserFind, AddressOf OnUserFind Plans.Add(Plan) Update() End Sub diff --git a/SCrawler/Download/TDownloader.vb b/SCrawler/Download/TDownloader.vb index 4b94e17..68936a4 100644 --- a/SCrawler/Download/TDownloader.vb +++ b/SCrawler/Download/TDownloader.vb @@ -8,21 +8,44 @@ ' but WITHOUT ANY WARRANTY Imports System.Threading Imports PersonalUtilities.Tools -Imports SCrawler.API Imports SCrawler.API.Base Imports SCrawler.Plugin.Hosts Imports Download = SCrawler.Plugin.ISiteSettings.Download -Imports EOptions = PersonalUtilities.Forms.Toolbars.IMyProgress.EnableOptions Namespace DownloadObjects Friend Class TDownloader : Implements IDisposable #Region "Events" - Friend Event OnJobsChange(ByVal JobsCount As Integer) - Friend Event OnDownloadCountChange() - Friend Event OnDownloading(ByVal Value As Boolean) + Friend Event JobsChange(ByVal JobsCount As Integer) + Friend Event DownloadCountChange() + Friend Event Downloading(ByVal Value As Boolean) Friend Event SendNotification As NotificationEventHandler - Friend Event OnReconfigured() + Friend Event Reconfigured() + Friend Event FeedFilesChanged(ByVal Added As Boolean) #End Region #Region "Declarations" +#Region "Files" + Friend Structure UserMediaD : Implements IComparable(Of UserMediaD), IEquatable(Of UserMediaD) + Friend ReadOnly User As IUserData + Friend ReadOnly Data As UserMedia + Friend ReadOnly [Date] As Date + Friend Sub New(ByVal Data As UserMedia, ByVal User As IUserData) + Me.Data = Data + Me.User = User + [Date] = Now + End Sub + Private Function CompareTo(ByVal Other As UserMediaD) As Integer Implements IComparable(Of UserMediaD).CompareTo + Return [Date].Ticks.CompareTo(Other.Date.Ticks) * -1 + End Function + Private Overloads Function Equals(ByVal Other As UserMediaD) As Boolean Implements IEquatable(Of UserMediaD).Equals + Return Data.File = Other.Data.File + End Function + Public Overloads Overrides Function Equals(ByVal Obj As Object) As Boolean + Return Equals(DirectCast(Obj, UserMedia)) + End Function + End Structure + Friend ReadOnly Property Files As List(Of UserMediaD) + Friend Property FilesChanged As Boolean = False + Private ReadOnly FilesLP As New ListAddParams(LAP.NotContainsOnly) +#End Region Friend ReadOnly Property Downloaded As List(Of IUserData) Private ReadOnly NProv As IFormatProvider #End Region @@ -49,7 +72,7 @@ Namespace DownloadObjects End Set End Property Friend Sub InvokeDownloadsChangeEvent() - RaiseEvent OnDownloadCountChange() + RaiseEvent DownloadCountChange() End Sub #End Region #Region "Jobs" @@ -163,6 +186,7 @@ Namespace DownloadObjects #End Region #Region "Initializer" Friend Sub New() + Files = New List(Of UserMediaD) Downloaded = New List(Of IUserData) NProv = New ANumbers With {.FormatOptions = ANumbers.Options.GroupIntegral} Pool = New List(Of Job) @@ -183,27 +207,30 @@ Namespace DownloadObjects End If Next End If - RaiseEvent OnReconfigured() + RaiseEvent Reconfigured() End If End Sub #End Region #Region "Thread" Private CheckerThread As Thread + Private MissingPostsDetected As Boolean = False Private Sub [Start]() If Not AutoDownloaderWorking AndAlso MyProgressForm.ReadyToOpen AndAlso Pool.LongCount(Function(p) p.Count > 0) > 1 Then MyProgressForm.Show() : MainFrameObj.Focus() If Not If(CheckerThread?.IsAlive, False) Then - MainProgress.Enabled = True + MainProgress.Visible = True If Not AutoDownloaderWorking AndAlso InfoForm.ReadyToOpen Then InfoForm.Show() : MainFrameObj.Focus() + MissingPostsDetected = False CheckerThread = New Thread(New ThreadStart(AddressOf JobsChecker)) CheckerThread.SetApartmentState(ApartmentState.MTA) CheckerThread.Start() End If - End Sub + End Sub Private Sub JobsChecker() - RaiseEvent OnDownloading(True) + Dim fBefore% = Files.Count + RaiseEvent Downloading(True) Try - MainProgress.TotalCount = 0 - MainProgress.CurrentCounter = 0 + MainProgress.Maximum = 0 + MainProgress.Value = 0 MyProgressForm.DisableProgressChange = False Do While Pool.Exists(Function(p) p.Count > 0 Or p.Working) For Each j As Job In Pool @@ -214,16 +241,23 @@ Namespace DownloadObjects Catch Finally With MainProgress - .TotalCount = 0 - .CurrentCounter = 0 + .Maximum = 0 + .Value = 0 .InformationTemporary = "All data downloaded" - .Enabled(EOptions.ProgressBar) = False + .Visible(, False) = False End With MyProgressForm.DisableProgressChange = True - If Pool.Count > 0 Then Pool.ForEach(Sub(p) If Not p.Progress Is Nothing Then p.Progress.TotalCount = 0) + If Pool.Count > 0 Then Pool.ForEach(Sub(p) If Not p.Progress Is Nothing Then p.Progress.Maximum = 0) ExecuteCommand(Settings.DownloadsCompleteCommand) UpdateJobsLabel() - RaiseEvent OnDownloading(False) + If MissingPostsDetected And Settings.AddMissingToLog Then + MyMainLOG = "Some posts didn't download. You can see them in the 'Missing posts' form." + MainFrameObj.UpdateLogButton() + End If + Files.Sort() + FilesChanged = Not fBefore = Files.Count + RaiseEvent Downloading(False) + If FilesChanged Then RaiseEvent FeedFilesChanged(True) End Try End Sub Private Sub StartDownloading(ByRef _Job As Job) @@ -236,9 +270,9 @@ Namespace DownloadObjects End Function Try _Job.Start() - _Job.Progress.TotalCount = 0 - _Job.Progress.CurrentCounter = 0 - _Job.Progress.Enabled = True + _Job.Progress.Maximum = 0 + _Job.Progress.Value = 0 + _Job.Progress.Visible = True Dim SiteChecked As Boolean = False Do While _Job.Count > 0 _Job.ThrowIfCancellationRequested() @@ -263,7 +297,7 @@ Namespace DownloadObjects If Pool.Count > 0 Then Pool.ForEach(Sub(j) If j.Working Then j.Stop()) End Sub Private Sub UpdateJobsLabel() - RaiseEvent OnJobsChange(Count) + RaiseEvent JobsChange(Count) End Sub Private Sub DownloadData(ByRef _Job As Job, ByVal Token As CancellationToken) Try @@ -291,7 +325,7 @@ Namespace DownloadObjects Next If t.Count > 0 Or Keys.Count > 0 Then With _Job.Progress - .Enabled(EOptions.All) = True + .Visible = True .Information = IIf(_Job.IsSeparated, $"{_Job.Name} d", "D") .Information &= $"ownloading {t.Count.NumToString(nf, NProv)}/{_Job.Items.Count.NumToString(nf, NProv)} profiles' data" .InformationTemporary = .Information @@ -303,9 +337,13 @@ Namespace DownloadObjects i = _Job.Items.FindIndex(Function(ii) ii.Key = k) If i >= 0 Then With _Job.Items(i) + If DirectCast(.Self, UserDataBase).ContentMissingExists Then MissingPostsDetected = True host.AfterDownload(_Job.Items(i), Download.Main) If Not .Disposed AndAlso Not .IsCollection AndAlso .DownloadedTotal(False) > 0 Then - If Not Downloaded.Contains(.Self) Then Downloaded.Add(GetUserFromMainCollection(.Self)) + If Not Downloaded.Contains(.Self) Then Downloaded.Add(Settings.GetUser(.Self)) + With DirectCast(.Self, UserDataBase) + If .LatestData.Count > 0 Then Files.ListAddList(.LatestData.Select(Function(d) New UserMediaD(d, .Self)), FilesLP) + End With dcc = True End If End With @@ -316,7 +354,7 @@ Namespace DownloadObjects Keys.Clear() _Job.Items.RemoveAll(Function(ii) ii.Disposed) If dcc Then Downloaded.RemoveAll(Function(u) u Is Nothing) - If dcc And Downloaded.Count > 0 Then RaiseEvent OnDownloadCountChange() + If dcc And Downloaded.Count > 0 Then RaiseEvent DownloadCountChange() t.Clear() End If End If @@ -332,30 +370,6 @@ Namespace DownloadObjects End Sub)) End Try End Sub - Friend Shared Function GetUserFromMainCollection(ByVal User As IUserData) As IUserData - Dim uSimple As Predicate(Of IUserData) = Function(u) u.Equals(DirectCast(User, UserDataBase)) - Dim uCol As Predicate(Of IUserData) = Function(ByVal u As IUserData) As Boolean - If u.IsCollection Then - Return DirectCast(u, UserDataBind).Collections.Exists(uSimple) - Else - Return False - End If - End Function - Dim uu As Predicate(Of IUserData) - If User.IncludedInCollection Then uu = uCol Else uu = uSimple - Dim i% = Settings.Users.FindIndex(uu) - If i >= 0 Then - If Settings.Users(i).IsCollection Then - With DirectCast(Settings.Users(i), UserDataBind) - i = .Collections.FindIndex(uSimple) - If i >= 0 Then Return .Collections(i) - End With - Else - Return Settings.Users(i) - End If - End If - Return Nothing - End Function #End Region #Region "Add" Private Sub AddItem(ByVal Item As IUserData, ByVal _UpdateJobsLabel As Boolean) @@ -395,7 +409,8 @@ Namespace DownloadObjects Return False End Function Friend Sub UserRemove(ByVal _Item As IUserData) - If Downloaded.Count > 0 AndAlso Downloaded.Contains(_Item) Then Downloaded.Remove(_Item) : RaiseEvent OnDownloadCountChange() + If Downloaded.Count > 0 AndAlso Downloaded.Contains(_Item) Then Downloaded.Remove(_Item) : RaiseEvent DownloadCountChange() + If Files.Count > 0 AndAlso Files.RemoveAll(Function(f) Not f.User Is Nothing AndAlso f.User.Equals(_Item)) > 0 Then RaiseEvent FeedFilesChanged(False) End Sub #End Region #Region "IDisposable Support" @@ -405,6 +420,7 @@ Namespace DownloadObjects If disposing Then [Stop]() Pool.ListClearDispose + Files.Clear() Downloaded.Clear() End If disposedValue = True diff --git a/SCrawler/Download/VideosDownloaderForm.vb b/SCrawler/Download/VideosDownloaderForm.vb index 0e1011b..33b814c 100644 --- a/SCrawler/Download/VideosDownloaderForm.vb +++ b/SCrawler/Download/VideosDownloaderForm.vb @@ -124,8 +124,8 @@ Namespace DownloadObjects Private Sub DownloadVideos() MyJob.Start() If MyJob.Count > 0 Then - MyJob.Progress.TotalCount = MyJob.Count - MyJob.Progress.Enabled = True + MyJob.Progress.Maximum = MyJob.Count + MyJob.Progress.Visible = True Dim IsFirst As Boolean = True For i% = MyJob.Count - 1 To 0 Step -1 If MyJob.IsCancellationRequested Then Exit For @@ -135,7 +135,7 @@ Namespace DownloadObjects Next MyJob.Progress.Done() RefillList() - MyJob.Progress.Enabled = False + MyJob.Progress.Visible = False End If SetControlValueInvoke(ToolbarTOP, BTT_DOWN, Sub() BTT_DOWN.Enabled = True) SetControlValueInvoke(ToolbarTOP, BTT_STOP, Sub() BTT_STOP.Enabled = False) diff --git a/SCrawler/Editors/GlobalSettingsForm.Designer.vb b/SCrawler/Editors/GlobalSettingsForm.Designer.vb index 68dfefd..d7b38cf 100644 --- a/SCrawler/Editors/GlobalSettingsForm.Designer.vb +++ b/SCrawler/Editors/GlobalSettingsForm.Designer.vb @@ -44,6 +44,10 @@ Dim TP_DOWNLOADING As System.Windows.Forms.TableLayoutPanel Dim ActionButton9 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton() Dim ActionButton10 As PersonalUtilities.Forms.Controls.Base.ActionButton = New PersonalUtilities.Forms.Controls.Base.ActionButton() + Dim TP_MISSING_DATA As System.Windows.Forms.TableLayoutPanel + Dim TAB_FEED As System.Windows.Forms.TabPage + Dim TP_FEED As System.Windows.Forms.TableLayoutPanel + Dim TP_FEED_IMG_COUNT As System.Windows.Forms.TableLayoutPanel Me.TXT_GLOBAL_PATH = New PersonalUtilities.Forms.Controls.TextBoxExtended() Me.TXT_IMAGE_LARGE = New PersonalUtilities.Forms.Controls.TextBoxExtended() Me.TXT_IMAGE_SMALL = New PersonalUtilities.Forms.Controls.TextBoxExtended() @@ -72,6 +76,8 @@ Me.CH_UDESCR_UP = New System.Windows.Forms.CheckBox() Me.CH_DOWN_OPEN_INFO_SUSPEND = New System.Windows.Forms.CheckBox() Me.CH_DOWN_OPEN_PROGRESS_SUSPEND = New System.Windows.Forms.CheckBox() + Me.CH_ADD_MISSING_TO_LOG = New System.Windows.Forms.CheckBox() + Me.CH_ADD_MISSING_ERROS_TO_LOG = New System.Windows.Forms.CheckBox() Me.TXT_CHANNELS_ROWS = New PersonalUtilities.Forms.Controls.TextBoxExtended() Me.TXT_CHANNELS_COLUMNS = New PersonalUtilities.Forms.Controls.TextBoxExtended() Me.CH_DOWN_IMAGES_NATIVE = New System.Windows.Forms.CheckBox() @@ -86,6 +92,9 @@ Me.CH_DOWN_OPEN_PROGRESS = New System.Windows.Forms.CheckBox() Me.TXT_SCRIPT = New PersonalUtilities.Forms.Controls.TextBoxExtended() Me.TXT_DOWN_COMPLETE_SCRIPT = New PersonalUtilities.Forms.Controls.TextBoxExtended() + Me.TXT_FEED_ROWS = New PersonalUtilities.Forms.Controls.TextBoxExtended() + Me.TXT_FEED_COLUMNS = New PersonalUtilities.Forms.Controls.TextBoxExtended() + Me.CH_FEED_ENDLESS = New System.Windows.Forms.CheckBox() Me.TAB_MAIN = New System.Windows.Forms.TabControl() Me.CONTAINER_MAIN = New System.Windows.Forms.ToolStripContainer() TP_BASIS = New System.Windows.Forms.TableLayoutPanel() @@ -106,6 +115,10 @@ TP_OPEN_PROGRESS = New System.Windows.Forms.TableLayoutPanel() TAB_DOWN = New System.Windows.Forms.TabPage() TP_DOWNLOADING = New System.Windows.Forms.TableLayoutPanel() + TP_MISSING_DATA = New System.Windows.Forms.TableLayoutPanel() + TAB_FEED = New System.Windows.Forms.TabPage() + TP_FEED = New System.Windows.Forms.TableLayoutPanel() + TP_FEED_IMG_COUNT = New System.Windows.Forms.TableLayoutPanel() TP_BASIS.SuspendLayout() CType(Me.TXT_GLOBAL_PATH, System.ComponentModel.ISupportInitialize).BeginInit() TP_IMAGES.SuspendLayout() @@ -136,6 +149,12 @@ TP_DOWNLOADING.SuspendLayout() CType(Me.TXT_SCRIPT, System.ComponentModel.ISupportInitialize).BeginInit() CType(Me.TXT_DOWN_COMPLETE_SCRIPT, System.ComponentModel.ISupportInitialize).BeginInit() + TP_MISSING_DATA.SuspendLayout() + TAB_FEED.SuspendLayout() + TP_FEED.SuspendLayout() + TP_FEED_IMG_COUNT.SuspendLayout() + CType(Me.TXT_FEED_ROWS, System.ComponentModel.ISupportInitialize).BeginInit() + CType(Me.TXT_FEED_COLUMNS, System.ComponentModel.ISupportInitialize).BeginInit() Me.TAB_MAIN.SuspendLayout() Me.CONTAINER_MAIN.ContentPanel.SuspendLayout() Me.CONTAINER_MAIN.SuspendLayout() @@ -606,6 +625,30 @@ TT_MAIN.SetToolTip(Me.CH_DOWN_OPEN_PROGRESS_SUSPEND, "Do not open the form automatically if it was once closed") Me.CH_DOWN_OPEN_PROGRESS_SUSPEND.UseVisualStyleBackColor = True ' + 'CH_ADD_MISSING_TO_LOG + ' + Me.CH_ADD_MISSING_TO_LOG.AutoSize = True + Me.CH_ADD_MISSING_TO_LOG.Dock = System.Windows.Forms.DockStyle.Fill + Me.CH_ADD_MISSING_TO_LOG.Location = New System.Drawing.Point(4, 4) + Me.CH_ADD_MISSING_TO_LOG.Name = "CH_ADD_MISSING_TO_LOG" + Me.CH_ADD_MISSING_TO_LOG.Size = New System.Drawing.Size(279, 17) + Me.CH_ADD_MISSING_TO_LOG.TabIndex = 0 + Me.CH_ADD_MISSING_TO_LOG.Text = "Add 'missing' information to log" + TT_MAIN.SetToolTip(Me.CH_ADD_MISSING_TO_LOG, resources.GetString("CH_ADD_MISSING_TO_LOG.ToolTip")) + Me.CH_ADD_MISSING_TO_LOG.UseVisualStyleBackColor = True + ' + 'CH_ADD_MISSING_ERROS_TO_LOG + ' + Me.CH_ADD_MISSING_ERROS_TO_LOG.AutoSize = True + Me.CH_ADD_MISSING_ERROS_TO_LOG.Dock = System.Windows.Forms.DockStyle.Fill + Me.CH_ADD_MISSING_ERROS_TO_LOG.Location = New System.Drawing.Point(290, 4) + Me.CH_ADD_MISSING_ERROS_TO_LOG.Name = "CH_ADD_MISSING_ERROS_TO_LOG" + Me.CH_ADD_MISSING_ERROS_TO_LOG.Size = New System.Drawing.Size(280, 17) + Me.CH_ADD_MISSING_ERROS_TO_LOG.TabIndex = 1 + Me.CH_ADD_MISSING_ERROS_TO_LOG.Text = "Add 'missing' errors to log" + TT_MAIN.SetToolTip(Me.CH_ADD_MISSING_ERROS_TO_LOG, resources.GetString("CH_ADD_MISSING_ERROS_TO_LOG.ToolTip")) + Me.CH_ADD_MISSING_ERROS_TO_LOG.UseVisualStyleBackColor = True + ' 'TP_CHANNELS_IMGS ' TP_CHANNELS_IMGS.ColumnCount = 2 @@ -962,15 +1005,17 @@ TP_DOWNLOADING.Controls.Add(Me.TXT_SCRIPT, 0, 3) TP_DOWNLOADING.Controls.Add(Me.CH_UDESCR_UP, 0, 0) TP_DOWNLOADING.Controls.Add(Me.TXT_DOWN_COMPLETE_SCRIPT, 0, 4) + TP_DOWNLOADING.Controls.Add(TP_MISSING_DATA, 0, 5) TP_DOWNLOADING.Dock = System.Windows.Forms.DockStyle.Fill TP_DOWNLOADING.Location = New System.Drawing.Point(0, 0) TP_DOWNLOADING.Name = "TP_DOWNLOADING" - TP_DOWNLOADING.RowCount = 6 + TP_DOWNLOADING.RowCount = 7 TP_DOWNLOADING.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!)) TP_DOWNLOADING.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 30.0!)) TP_DOWNLOADING.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 30.0!)) TP_DOWNLOADING.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 28.0!)) TP_DOWNLOADING.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 28.0!)) + TP_DOWNLOADING.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!)) TP_DOWNLOADING.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Percent, 100.0!)) TP_DOWNLOADING.Size = New System.Drawing.Size(576, 284) TP_DOWNLOADING.TabIndex = 0 @@ -1012,6 +1057,111 @@ Me.TXT_DOWN_COMPLETE_SCRIPT.Size = New System.Drawing.Size(568, 22) Me.TXT_DOWN_COMPLETE_SCRIPT.TabIndex = 4 ' + 'TP_MISSING_DATA + ' + TP_MISSING_DATA.CellBorderStyle = System.Windows.Forms.TableLayoutPanelCellBorderStyle.[Single] + TP_MISSING_DATA.ColumnCount = 2 + TP_MISSING_DATA.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Percent, 50.0!)) + TP_MISSING_DATA.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Percent, 50.0!)) + TP_MISSING_DATA.Controls.Add(Me.CH_ADD_MISSING_TO_LOG, 0, 0) + TP_MISSING_DATA.Controls.Add(Me.CH_ADD_MISSING_ERROS_TO_LOG, 1, 0) + TP_MISSING_DATA.Dock = System.Windows.Forms.DockStyle.Fill + TP_MISSING_DATA.Location = New System.Drawing.Point(1, 147) + TP_MISSING_DATA.Margin = New System.Windows.Forms.Padding(0) + TP_MISSING_DATA.Name = "TP_MISSING_DATA" + TP_MISSING_DATA.RowCount = 1 + TP_MISSING_DATA.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Percent, 100.0!)) + TP_MISSING_DATA.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 24.0!)) + TP_MISSING_DATA.Size = New System.Drawing.Size(574, 25) + TP_MISSING_DATA.TabIndex = 5 + ' + 'TAB_FEED + ' + TAB_FEED.Controls.Add(TP_FEED) + TAB_FEED.Location = New System.Drawing.Point(4, 22) + TAB_FEED.Name = "TAB_FEED" + TAB_FEED.Size = New System.Drawing.Size(576, 284) + TAB_FEED.TabIndex = 7 + TAB_FEED.Text = "Feed" + ' + 'TP_FEED + ' + TP_FEED.CellBorderStyle = System.Windows.Forms.TableLayoutPanelCellBorderStyle.[Single] + TP_FEED.ColumnCount = 1 + TP_FEED.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Percent, 100.0!)) + TP_FEED.Controls.Add(TP_FEED_IMG_COUNT, 0, 0) + TP_FEED.Controls.Add(Me.CH_FEED_ENDLESS, 0, 1) + TP_FEED.Dock = System.Windows.Forms.DockStyle.Fill + TP_FEED.Location = New System.Drawing.Point(0, 0) + TP_FEED.Name = "TP_FEED" + TP_FEED.RowCount = 3 + TP_FEED.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 28.0!)) + TP_FEED.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!)) + TP_FEED.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Percent, 100.0!)) + TP_FEED.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 20.0!)) + TP_FEED.Size = New System.Drawing.Size(576, 284) + TP_FEED.TabIndex = 0 + ' + 'TP_FEED_IMG_COUNT + ' + TP_FEED_IMG_COUNT.ColumnCount = 2 + TP_FEED_IMG_COUNT.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Percent, 50.0!)) + TP_FEED_IMG_COUNT.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Percent, 50.0!)) + TP_FEED_IMG_COUNT.Controls.Add(Me.TXT_FEED_ROWS, 0, 0) + TP_FEED_IMG_COUNT.Controls.Add(Me.TXT_FEED_COLUMNS, 1, 0) + TP_FEED_IMG_COUNT.Dock = System.Windows.Forms.DockStyle.Fill + TP_FEED_IMG_COUNT.Location = New System.Drawing.Point(1, 1) + TP_FEED_IMG_COUNT.Margin = New System.Windows.Forms.Padding(0) + TP_FEED_IMG_COUNT.Name = "TP_FEED_IMG_COUNT" + TP_FEED_IMG_COUNT.RowCount = 1 + TP_FEED_IMG_COUNT.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Percent, 100.0!)) + TP_FEED_IMG_COUNT.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 28.0!)) + TP_FEED_IMG_COUNT.Size = New System.Drawing.Size(574, 28) + TP_FEED_IMG_COUNT.TabIndex = 1 + ' + 'TXT_FEED_ROWS + ' + Me.TXT_FEED_ROWS.CaptionText = "Feed rows" + Me.TXT_FEED_ROWS.CaptionToolTipEnabled = True + Me.TXT_FEED_ROWS.CaptionToolTipText = "How many lines of images should be shown in the feed form" + Me.TXT_FEED_ROWS.ControlMode = PersonalUtilities.Forms.Controls.TextBoxExtended.ControlModes.NumericUpDown + Me.TXT_FEED_ROWS.Dock = System.Windows.Forms.DockStyle.Fill + Me.TXT_FEED_ROWS.Location = New System.Drawing.Point(3, 3) + Me.TXT_FEED_ROWS.Name = "TXT_FEED_ROWS" + Me.TXT_FEED_ROWS.NumberMaximum = New Decimal(New Integer() {50, 0, 0, 0}) + Me.TXT_FEED_ROWS.NumberMinimum = New Decimal(New Integer() {1, 0, 0, 0}) + Me.TXT_FEED_ROWS.Size = New System.Drawing.Size(281, 22) + Me.TXT_FEED_ROWS.TabIndex = 0 + Me.TXT_FEED_ROWS.Text = "1" + Me.TXT_FEED_ROWS.TextBoxTextAlign = System.Windows.Forms.HorizontalAlignment.Center + ' + 'TXT_FEED_COLUMNS + ' + Me.TXT_FEED_COLUMNS.CaptionText = "Feed columns" + Me.TXT_FEED_COLUMNS.CaptionToolTipEnabled = True + Me.TXT_FEED_COLUMNS.CaptionToolTipText = "How many columns of images should be shown in the feed form" + Me.TXT_FEED_COLUMNS.ControlMode = PersonalUtilities.Forms.Controls.TextBoxExtended.ControlModes.NumericUpDown + Me.TXT_FEED_COLUMNS.Dock = System.Windows.Forms.DockStyle.Fill + Me.TXT_FEED_COLUMNS.Location = New System.Drawing.Point(290, 3) + Me.TXT_FEED_COLUMNS.Name = "TXT_FEED_COLUMNS" + Me.TXT_FEED_COLUMNS.NumberMaximum = New Decimal(New Integer() {20, 0, 0, 0}) + Me.TXT_FEED_COLUMNS.NumberMinimum = New Decimal(New Integer() {1, 0, 0, 0}) + Me.TXT_FEED_COLUMNS.Size = New System.Drawing.Size(281, 22) + Me.TXT_FEED_COLUMNS.TabIndex = 1 + Me.TXT_FEED_COLUMNS.Text = "1" + Me.TXT_FEED_COLUMNS.TextBoxTextAlign = System.Windows.Forms.HorizontalAlignment.Center + ' + 'CH_FEED_ENDLESS + ' + Me.CH_FEED_ENDLESS.AutoSize = True + Me.CH_FEED_ENDLESS.Dock = System.Windows.Forms.DockStyle.Fill + Me.CH_FEED_ENDLESS.Location = New System.Drawing.Point(4, 33) + Me.CH_FEED_ENDLESS.Name = "CH_FEED_ENDLESS" + Me.CH_FEED_ENDLESS.Size = New System.Drawing.Size(568, 19) + Me.CH_FEED_ENDLESS.TabIndex = 3 + Me.CH_FEED_ENDLESS.Text = "Endless feed" + Me.CH_FEED_ENDLESS.UseVisualStyleBackColor = True + ' 'TAB_MAIN ' Me.TAB_MAIN.Controls.Add(TAB_BASIS) @@ -1019,6 +1169,7 @@ Me.TAB_MAIN.Controls.Add(TAB_DEFAULTS) Me.TAB_MAIN.Controls.Add(TAB_DOWN) Me.TAB_MAIN.Controls.Add(TAB_DEFS_CHANNELS) + Me.TAB_MAIN.Controls.Add(TAB_FEED) Me.TAB_MAIN.Dock = System.Windows.Forms.DockStyle.Fill Me.TAB_MAIN.Location = New System.Drawing.Point(0, 0) Me.TAB_MAIN.Name = "TAB_MAIN" @@ -1098,6 +1249,14 @@ TP_DOWNLOADING.PerformLayout() CType(Me.TXT_SCRIPT, System.ComponentModel.ISupportInitialize).EndInit() CType(Me.TXT_DOWN_COMPLETE_SCRIPT, System.ComponentModel.ISupportInitialize).EndInit() + TP_MISSING_DATA.ResumeLayout(False) + TP_MISSING_DATA.PerformLayout() + TAB_FEED.ResumeLayout(False) + TP_FEED.ResumeLayout(False) + TP_FEED.PerformLayout() + TP_FEED_IMG_COUNT.ResumeLayout(False) + CType(Me.TXT_FEED_ROWS, System.ComponentModel.ISupportInitialize).EndInit() + CType(Me.TXT_FEED_COLUMNS, System.ComponentModel.ISupportInitialize).EndInit() Me.TAB_MAIN.ResumeLayout(False) Me.CONTAINER_MAIN.ContentPanel.ResumeLayout(False) Me.CONTAINER_MAIN.ResumeLayout(False) @@ -1150,5 +1309,10 @@ Private WithEvents CH_DOWN_OPEN_INFO_SUSPEND As CheckBox Private WithEvents CH_DOWN_OPEN_PROGRESS_SUSPEND As CheckBox Private WithEvents CH_DOWN_IMAGES_NATIVE As CheckBox + Private WithEvents CH_ADD_MISSING_TO_LOG As CheckBox + Private WithEvents CH_ADD_MISSING_ERROS_TO_LOG As CheckBox + Private WithEvents TXT_FEED_ROWS As PersonalUtilities.Forms.Controls.TextBoxExtended + Private WithEvents TXT_FEED_COLUMNS As PersonalUtilities.Forms.Controls.TextBoxExtended + Private WithEvents CH_FEED_ENDLESS As CheckBox End Class End Namespace \ No newline at end of file diff --git a/SCrawler/Editors/GlobalSettingsForm.resx b/SCrawler/Editors/GlobalSettingsForm.resx index 6055126..a200979 100644 --- a/SCrawler/Editors/GlobalSettingsForm.resx +++ b/SCrawler/Editors/GlobalSettingsForm.resx @@ -210,6 +210,16 @@ This is a global setting for newly added users only. This parameter specifies how the video will be stored in the users' download path. If checked, videos will be stored in separate folder; otherwise, videos will be stored along with images. + + + Add information about missing posts (that were not downloaded) to the log. +That means that text like "Missing posts exist" will be added to the log. +You can find more detailed information about the missing posts in the form that opens by right-clicking on the 'Info' button in the main window. + + + Add error information about missing posts (that were not downloaded) to the log. +That means that the error text about the not downloaded post will be added to the log. +You can find more detailed information about the missing posts in the form that opens by right-clicking on the 'Info' button in the main window. False @@ -282,6 +292,18 @@ If checked, videos will be stored in separate folder; otherwise, videos will be AFuc5QFgn6ClHh5iOQVAKNixyucB8NY0vG9JOzzyhrdq5IRgAAAAAElFTkSuQmCC + + False + + + False + + + False + + + False + AAABAA8AAAAQAAEABAAwOgAA9gAAADAwEAABAAQAaAYAACg7AAAgIBAAAQAEAOgCAACQQQAAGBgQAAEA diff --git a/SCrawler/Editors/GlobalSettingsForm.vb b/SCrawler/Editors/GlobalSettingsForm.vb index e0b2ae0..dbb5a94 100644 --- a/SCrawler/Editors/GlobalSettingsForm.vb +++ b/SCrawler/Editors/GlobalSettingsForm.vb @@ -8,9 +8,11 @@ ' but WITHOUT ANY WARRANTY Imports PersonalUtilities.Forms Imports PersonalUtilities.Forms.Controls.Base +Imports ADB = PersonalUtilities.Forms.Controls.Base.ActionButton.DefaultButtons Namespace Editors Friend Class GlobalSettingsForm Private WithEvents MyDefs As DefaultFormOptions + Friend Property FeedParametersChanged As Boolean = False Friend Sub New() InitializeComponent() MyDefs = New DefaultFormOptions(Me, Settings.Design) @@ -58,6 +60,8 @@ Namespace Editors TXT_SCRIPT.Text = .ScriptData.Value TXT_DOWN_COMPLETE_SCRIPT.Text = .DownloadsCompleteCommand TXT_DOWN_COMPLETE_SCRIPT.Checked = .DownloadsCompleteCommand.Attribute + CH_ADD_MISSING_TO_LOG.Checked = .AddMissingToLog + CH_ADD_MISSING_ERROS_TO_LOG.Checked = .AddMissingErrorsToLog 'Downloading: file names CH_FILE_NAME_CHANGE.Checked = Not .FileReplaceNameByDate.Value = FileNameReplaceMode.None OPT_FILE_NAME_REPLACE.Checked = .FileReplaceNameByDate.Value = FileNameReplaceMode.Replace @@ -75,6 +79,10 @@ Namespace Editors CH_COPY_CHANNEL_USER_IMAGE_ALL.Checked = .ChannelsAddUserImagesFromAllChannels CH_COPY_CHANNEL_USER_IMAGE_ALL.Enabled = CH_COPY_CHANNEL_USER_IMAGE.Checked CH_CHANNELS_USERS_TEMP.Checked = .ChannelsDefaultTemporary + 'Feed + TXT_FEED_ROWS.Value = .FeedDataRows.Value + TXT_FEED_COLUMNS.Value = .FeedDataColumns.Value + CH_FEED_ENDLESS.Checked = .FeedEndless End With .MyFieldsChecker = New FieldsChecker With .MyFieldsCheckerE @@ -158,6 +166,8 @@ Namespace Editors .ScriptData.Attribute.Value = TXT_SCRIPT.Checked .DownloadsCompleteCommand.Value = TXT_DOWN_COMPLETE_SCRIPT.Text .DownloadsCompleteCommand.Attribute.Value = TXT_DOWN_COMPLETE_SCRIPT.Checked + .AddMissingToLog.Value = CH_ADD_MISSING_TO_LOG.Checked + .AddMissingErrorsToLog.Value = CH_ADD_MISSING_ERROS_TO_LOG.Checked 'Downloading: file names If CH_FILE_NAME_CHANGE.Checked Then .FileReplaceNameByDate.Value = If(OPT_FILE_NAME_REPLACE.Checked, FileNameReplaceMode.Replace, FileNameReplaceMode.Add) @@ -177,6 +187,11 @@ Namespace Editors .FromChannelCopyImageToUser.Value = CH_COPY_CHANNEL_USER_IMAGE.Checked .ChannelsAddUserImagesFromAllChannels.Value = CH_COPY_CHANNEL_USER_IMAGE_ALL.Checked .ChannelsDefaultTemporary.Value = CH_CHANNELS_USERS_TEMP.Checked + 'Feed + .FeedDataRows.Value = CInt(TXT_FEED_ROWS.Value) + .FeedDataColumns.Value = CInt(TXT_FEED_COLUMNS.Value) + .FeedEndless.Value = CH_FEED_ENDLESS.Checked + FeedParametersChanged = .FeedDataRows.ChangesDetected Or .FeedDataColumns.ChangesDetected Or .FeedEndless.ChangesDetected .EndUpdate() End With @@ -184,16 +199,16 @@ Namespace Editors End If End Sub Private Sub TXT_GLOBAL_PATH_ActionOnButtonClick(ByVal Sender As ActionButton, ByVal e As EventArgs) Handles TXT_GLOBAL_PATH.ActionOnButtonClick - If Sender.DefaultButton = ActionButton.DefaultButtons.Open Then + If Sender.DefaultButton = ADB.Open Then Dim f As SFile = SFile.SelectPath(Settings.GlobalPath.Value) If Not f.IsEmptyString Then TXT_GLOBAL_PATH.Text = f End If End Sub Private Sub TXT_MAX_JOBS_USERS_ActionOnButtonClick(ByVal Sender As ActionButton, ByVal e As EventArgs) Handles TXT_MAX_JOBS_USERS.ActionOnButtonClick - If Sender.DefaultButton = ActionButton.DefaultButtons.Refresh Then TXT_MAX_JOBS_USERS.Value = SettingsCLS.DefaultMaxDownloadingTasks + If Sender.DefaultButton = ADB.Refresh Then TXT_MAX_JOBS_USERS.Value = SettingsCLS.DefaultMaxDownloadingTasks End Sub Private Sub TXT_MAX_JOBS_CHANNELS_ActionOnButtonClick(ByVal Sender As ActionButton, ByVal e As EventArgs) Handles TXT_MAX_JOBS_CHANNELS.ActionOnButtonClick - If Sender.DefaultButton = ActionButton.DefaultButtons.Refresh Then TXT_MAX_JOBS_CHANNELS.Value = SettingsCLS.DefaultMaxDownloadingTasks + If Sender.DefaultButton = ADB.Refresh Then TXT_MAX_JOBS_CHANNELS.Value = SettingsCLS.DefaultMaxDownloadingTasks End Sub Private Sub CH_FILE_NAME_CHANGE_CheckedChanged(sender As Object, e As EventArgs) Handles CH_FILE_NAME_CHANGE.CheckedChanged ChangeFileNameChangersEnabling() diff --git a/SCrawler/EncryptCookies.vb b/SCrawler/EncryptCookies.vb new file mode 100644 index 0000000..a298d3e --- /dev/null +++ b/SCrawler/EncryptCookies.vb @@ -0,0 +1,28 @@ +' Copyright (C) 2022 Andy +' This program is free software: you can redistribute it and/or modify +' it under the terms of the GNU General Public License as published by +' the Free Software Foundation, either version 3 of the License, or +' (at your option) any later version. +' +' This program is distributed in the hope that it will be useful, +' but WITHOUT ANY WARRANTY +Imports PersonalUtilities.Tools.WEB +Namespace EncryptCookies + Friend Module EncryptFunction + Friend CookiesEncrypted As Boolean = False + Friend Sub ValidateCookiesEncrypt(ByRef Responser As Response) + If Not Responser Is Nothing Then + Dim b As Boolean = False + With Responser + If Not .Cookies Is Nothing Then + With .Cookies + If .EncryptKey.IsEmptyString Then .EncryptKey = SettingsCLS.CookieEncryptKey : b = .Count > 0 + End With + End If + If .CookiesEncryptKey.IsEmptyString Then .CookiesEncryptKey = SettingsCLS.CookieEncryptKey : b = True + If b Then .SaveSettings() + End With + End If + End Sub + End Module +End Namespace \ No newline at end of file diff --git a/SCrawler/MainFrame.Designer.vb b/SCrawler/MainFrame.Designer.vb index bfb44f1..c437364 100644 --- a/SCrawler/MainFrame.Designer.vb +++ b/SCrawler/MainFrame.Designer.vb @@ -105,6 +105,7 @@ Partial Public Class MainFrame : Inherits System.Windows.Forms.Form Me.BTT_TRAY_SHOW_HIDE = New System.Windows.Forms.ToolStripMenuItem() Me.BTT_TRAY_CLOSE = New System.Windows.Forms.ToolStripMenuItem() Me.BTT_TRAY_CLOSE_NO_SCRIPT = New System.Windows.Forms.ToolStripMenuItem() + Me.BTT_FEED = New System.Windows.Forms.ToolStripButton() SEP_1 = New System.Windows.Forms.ToolStripSeparator() SEP_2 = New System.Windows.Forms.ToolStripSeparator() CONTEXT_SEP_1 = New System.Windows.Forms.ToolStripSeparator() @@ -215,11 +216,12 @@ Partial Public Class MainFrame : Inherits System.Windows.Forms.Form ' 'MENU_SETTINGS ' - Me.MENU_SETTINGS.AutoToolTip = False + Me.MENU_SETTINGS.DisplayStyle = System.Windows.Forms.ToolStripItemDisplayStyle.Image Me.MENU_SETTINGS.DropDownItems.AddRange(New System.Windows.Forms.ToolStripItem() {MENU_SETTINGS_SEP_1, Me.BTT_SETTINGS}) + Me.MENU_SETTINGS.Image = Global.SCrawler.My.Resources.Resources.SettingsPic_16 Me.MENU_SETTINGS.ImageTransparentColor = System.Drawing.Color.Magenta Me.MENU_SETTINGS.Name = "MENU_SETTINGS" - Me.MENU_SETTINGS.Size = New System.Drawing.Size(62, 22) + Me.MENU_SETTINGS.Size = New System.Drawing.Size(29, 22) Me.MENU_SETTINGS.Text = "Settings" ' 'BTT_SETTINGS @@ -232,7 +234,7 @@ Partial Public Class MainFrame : Inherits System.Windows.Forms.Form 'Toolbar_TOP ' Me.Toolbar_TOP.GripStyle = System.Windows.Forms.ToolStripGripStyle.Hidden - Me.Toolbar_TOP.Items.AddRange(New System.Windows.Forms.ToolStripItem() {Me.MENU_SETTINGS, SEP_1, Me.BTT_ADD_USER, Me.BTT_EDIT_USER, Me.BTT_DELETE_USER, Me.BTT_REFRESH, Me.BTT_SHOW_INFO, Me.BTT_CHANNELS, Me.BTT_DOWN_SAVED, SEP_2, Me.BTT_DOWN_SELECTED, Me.MENU_DOWN_ALL, Me.BTT_DOWN_VIDEO, Me.BTT_DOWN_STOP, SEP_3, Me.MENU_VIEW, SEP_4, Me.BTT_LOG, Me.BTT_VERSION_INFO, Me.BTT_DONATE}) + Me.Toolbar_TOP.Items.AddRange(New System.Windows.Forms.ToolStripItem() {Me.MENU_SETTINGS, SEP_1, Me.BTT_ADD_USER, Me.BTT_EDIT_USER, Me.BTT_DELETE_USER, Me.BTT_REFRESH, Me.BTT_SHOW_INFO, Me.BTT_FEED, Me.BTT_CHANNELS, Me.BTT_DOWN_SAVED, SEP_2, Me.BTT_DOWN_SELECTED, Me.MENU_DOWN_ALL, Me.BTT_DOWN_VIDEO, Me.BTT_DOWN_STOP, SEP_3, Me.MENU_VIEW, SEP_4, Me.BTT_LOG, Me.BTT_VERSION_INFO, Me.BTT_DONATE}) Me.Toolbar_TOP.Location = New System.Drawing.Point(0, 0) Me.Toolbar_TOP.Name = "Toolbar_TOP" Me.Toolbar_TOP.Size = New System.Drawing.Size(934, 25) @@ -281,7 +283,8 @@ Partial Public Class MainFrame : Inherits System.Windows.Forms.Form Me.BTT_SHOW_INFO.Name = "BTT_SHOW_INFO" Me.BTT_SHOW_INFO.Size = New System.Drawing.Size(48, 22) Me.BTT_SHOW_INFO.Text = "Info" - Me.BTT_SHOW_INFO.ToolTipText = "Show download summary" + Me.BTT_SHOW_INFO.ToolTipText = "Left-click: open the 'Info' form (show download summary)." & Global.Microsoft.VisualBasic.ChrW(13) & Global.Microsoft.VisualBasic.ChrW(10) & "Right click: open the " & + "'Missing' form (show information about missing posts)." ' 'BTT_CHANNELS ' @@ -773,6 +776,14 @@ Partial Public Class MainFrame : Inherits System.Windows.Forms.Form Me.BTT_TRAY_CLOSE_NO_SCRIPT.ToolTipText = "Close the program without executing the script" Me.BTT_TRAY_CLOSE_NO_SCRIPT.Visible = False ' + 'BTT_FEED + ' + Me.BTT_FEED.Image = Global.SCrawler.My.Resources.Resources.RSSPic + Me.BTT_FEED.ImageTransparentColor = System.Drawing.Color.Magenta + Me.BTT_FEED.Name = "BTT_FEED" + Me.BTT_FEED.Size = New System.Drawing.Size(52, 22) + Me.BTT_FEED.Text = "Feed" + ' 'MainFrame ' Me.AutoScaleDimensions = New System.Drawing.SizeF(6.0!, 13.0!) @@ -801,7 +812,6 @@ Partial Public Class MainFrame : Inherits System.Windows.Forms.Form Private WithEvents BTT_DELETE_USER As ToolStripButton Private WithEvents BTT_DOWN_SELECTED As ToolStripButton Private WithEvents MENU_DOWN_ALL As ToolStripDropDownButton - Private WithEvents Toolbar_TOP As ToolStrip Private WithEvents Toolbar_BOTTOM As StatusStrip Private WithEvents PR_MAIN As ToolStripProgressBar Private WithEvents LBL_STATUS As ToolStripStatusLabel @@ -823,7 +833,6 @@ Partial Public Class MainFrame : Inherits System.Windows.Forms.Form Private WithEvents LBL_JOBS_COUNT As ToolStripStatusLabel Private WithEvents BTT_DOWN_VIDEO As ToolStripButton Private WithEvents BTT_SHOW_INFO As ToolStripButton - Private WithEvents BTT_LOG As ToolStripButton Private WithEvents BTT_CHANNELS As ToolStripButton Private WithEvents LIST_PROFILES As ListView Private WithEvents MENU_VIEW As ToolStripDropDownButton @@ -869,4 +878,7 @@ Partial Public Class MainFrame : Inherits System.Windows.Forms.Form Friend WithEvents TRAY_CONTEXT As ContextMenuStrip Private WithEvents BTT_DOWN_AUTOMATION As ToolStripMenuItem Private WithEvents BTT_DOWN_AUTOMATION_PAUSE As ToolStripMenuItem + Friend WithEvents BTT_LOG As ToolStripButton + Friend WithEvents Toolbar_TOP As ToolStrip + Private WithEvents BTT_FEED As ToolStripButton End Class \ No newline at end of file diff --git a/SCrawler/MainFrame.vb b/SCrawler/MainFrame.vb index bc56a72..32cc6eb 100644 --- a/SCrawler/MainFrame.vb +++ b/SCrawler/MainFrame.vb @@ -20,6 +20,8 @@ Public Class MainFrame Private ReadOnly _VideoDownloadingMode As Boolean = False Private MyChannels As ChannelViewForm Private MySavedPosts As DownloadSavedPostsForm + Private MyMissingPosts As MissingPostsForm + Private MyFeed As DownloadFeedForm Private _UFinit As Boolean = True Public Sub New() InitializeComponent() @@ -43,19 +45,19 @@ Public Class MainFrame End Sub Private Sub MainFrame_Load(sender As Object, e As EventArgs) Handles Me.Load If _VideoDownloadingMode Then GoTo FormClosingInvoker - If Now.Month = 6 Then Text = "SCrawler: Happy LGBT Pride Month! :-)" + If Now.Month.ValueBetween(6, 8) Then Text = "SCrawler: Happy LGBT Pride Month! :-)" Settings.DeleteCachePath() MainFrameObj = New MainFrameObjects(Me) MainFrameObj.ChangeCloseVisible() MainProgress = New Toolbars.MyProgress(Toolbar_BOTTOM, PR_MAIN, LBL_STATUS, "Downloading profiles' data") With { - .DropCurrentProgressOnTotalChange = False, .Enabled = False} + .ResetProgressOnMaximumChanges = False, .Visible = False} Downloader = New TDownloader InfoForm = New DownloadedInfoForm MyProgressForm = New ActiveDownloadingProgress Downloader.ReconfPool() - AddHandler Downloader.OnJobsChange, AddressOf Downloader_UpdateJobsCount - AddHandler Downloader.OnDownloading, AddressOf Downloader_OnDownloading - AddHandler Downloader.OnDownloadCountChange, AddressOf InfoForm.Downloader_OnDownloadCountChange + AddHandler Downloader.JobsChange, AddressOf Downloader_UpdateJobsCount + AddHandler Downloader.Downloading, AddressOf Downloader_Downloading + AddHandler Downloader.DownloadCountChange, AddressOf InfoForm.Downloader_DownloadCountChange AddHandler Downloader.SendNotification, AddressOf NotificationMessage AddHandler InfoForm.UserFind, AddressOf FocusUser Settings.LoadUsers() @@ -97,7 +99,6 @@ Public Class MainFrame Settings.Automation = New Scheduler AddHandler Settings.Groups.Updated, AddressOf Settings.Automation.GROUPS_Updated AddHandler Settings.Groups.Deleted, AddressOf Settings.Automation.GROUPS_Deleted - AddHandler Settings.Automation.UserFind, AddressOf FocusUser _UFinit = False Settings.Automation.Start(True) GoTo EndFunction @@ -138,6 +139,8 @@ EndFunction: Downloader.Dispose() MyProgressForm.Dispose() InfoForm.Dispose() + If Not MyMissingPosts Is Nothing Then MyMissingPosts.Dispose() + If Not MyFeed Is Nothing Then MyFeed.Dispose() MainFrameObj.ClearNotifications() If Not MyChannels Is Nothing Then MyChannels.Dispose() If Not VideoDownloader Is Nothing Then VideoDownloader.Dispose() @@ -272,6 +275,7 @@ CloseResume: (Not sg = Settings.ShowGroups And .UseGrouping) Then RefillList() TrayIcon.Visible = .CloseToTray LIST_PROFILES.ShowGroups = .UseGrouping + If f.FeedParametersChanged And Not MyFeed Is Nothing Then MyFeed.UpdateSettings() End If End Using End With @@ -335,6 +339,16 @@ CloseResume: Private Sub BTT_SHOW_INFO_Click(sender As Object, e As EventArgs) Handles BTT_SHOW_INFO.Click ShowInfoForm(True) End Sub + Private Sub BTT_SHOW_INFO_MouseDown(sender As Object, e As MouseEventArgs) Handles BTT_SHOW_INFO.MouseDown + If e.Button = MouseButtons.Right Then + If MyMissingPosts Is Nothing Then MyMissingPosts = New MissingPostsForm + If MyMissingPosts.Visible Then MyMissingPosts.BringToFront() Else MyMissingPosts.Show() + End If + End Sub + Private Sub BTT_FEED_Click(sender As Object, e As EventArgs) Handles BTT_FEED.Click + If MyFeed Is Nothing Then MyFeed = New DownloadFeedForm : AddHandler Downloader.FeedFilesChanged, AddressOf MyFeed.Downloader_FilesChanged + If MyFeed.Visible Then MyFeed.BringToFront() Else MyFeed.Show() + End Sub Private Overloads Sub ShowInfoForm() ShowInfoForm(False) End Sub @@ -356,7 +370,7 @@ CloseResume: Private Sub BTT_DOWN_SAVED_Click(sender As Object, e As EventArgs) Handles BTT_DOWN_SAVED.Click If MySavedPosts Is Nothing Then MySavedPosts = New DownloadSavedPostsForm - AddHandler MySavedPosts.OnDownloadDone, AddressOf NotificationMessage + AddHandler MySavedPosts.DownloadDone, AddressOf NotificationMessage End If With MySavedPosts If .Visible Then .BringToFront() Else .Show() @@ -605,7 +619,7 @@ CloseResume: End Sub #End Region Private Sub BTT_LOG_Click(sender As Object, e As EventArgs) Handles BTT_LOG.Click - MyMainLOG_ShowForm(Settings.Design) + MyMainLOG_ShowForm(Settings.Design,,,, Sub() MainFrameObj.UpdateLogButton()) End Sub Private Sub BTT_DONATE_Click(sender As Object, e As EventArgs) Handles BTT_DONATE.Click Try : Process.Start("https://github.com/AAndyProgram/SCrawler/blob/main/HowToSupport.md") : Catch : End Try @@ -762,7 +776,7 @@ CloseResume: Dim Added As Boolean = i < 0 If i < 0 Then .Users.Add(New UserDataBind(f.Collection)) - CollectionHandler(DirectCast(.Users.Last, UserDataBind)) + MainFrameObj.CollectionHandler(DirectCast(.Users.Last, UserDataBind)) i = .Users.Count - 1 End If Try @@ -1153,7 +1167,7 @@ ResumeDownloadingOperation: Private Overloads Sub FocusUser(ByVal Key As String) FocusUser(Key, True) End Sub - Private Overloads Sub FocusUser(ByVal Key As String, Optional ByVal ActivateMe As Boolean = False) + Friend Overloads Sub FocusUser(ByVal Key As String, Optional ByVal ActivateMe As Boolean = False) Dim a As Action = Sub() Dim i% = LIST_PROFILES.Items.IndexOfKey(Key) If i < 0 Then @@ -1183,16 +1197,14 @@ ResumeDownloadingOperation: Dim a As Action = Sub() LBL_JOBS_COUNT.Text = IIf(TotalCount = 0, String.Empty, $"[Jobs {TotalCount}]") If Toolbar_BOTTOM.InvokeRequired Then Toolbar_BOTTOM.Invoke(a) Else a.Invoke If Not _LogColorChanged AndAlso Not MyMainLOG.IsEmptyString Then - a = Sub() BTT_LOG.ControlChangeColor(False) - If Toolbar_TOP.InvokeRequired Then Toolbar_TOP.Invoke(a) Else a.Invoke + MainFrameObj.UpdateLogButton() _LogColorChanged = True ElseIf _LogColorChanged And MyMainLOG.IsEmptyString Then - a = Sub() BTT_LOG.ControlChangeColor(SystemColors.Control, SystemColors.ControlText) - If Toolbar_TOP.InvokeRequired Then Toolbar_TOP.Invoke(a) Else a.Invoke + MainFrameObj.UpdateLogButton() _LogColorChanged = False End If End Sub - Private Sub Downloader_OnDownloading(ByVal Value As Boolean) + Private Sub Downloader_Downloading(ByVal Value As Boolean) Dim a As Action = Sub() BTT_DOWN_STOP.Enabled = Value Or Downloader.Working If Toolbar_TOP.InvokeRequired Then Toolbar_TOP.Invoke(a) Else a.Invoke End Sub diff --git a/SCrawler/MainFrameObjects.vb b/SCrawler/MainFrameObjects.vb index 3ea0ece..478c06f 100644 --- a/SCrawler/MainFrameObjects.vb +++ b/SCrawler/MainFrameObjects.vb @@ -16,6 +16,12 @@ Friend Class MainFrameObjects MF = f Notificator = New NotificationsManager End Sub +#Region "Users" + Friend Sub FocusUser(ByVal Key As String, Optional ByVal ActivateForm As Boolean = False) + MF.FocusUser(Key, ActivateForm) + End Sub +#End Region +#Region "Image handlers" Friend Sub ImageHandler(ByVal User As IUserData) ImageHandler(User, False) ImageHandler(User, True) @@ -37,6 +43,8 @@ Friend Class MainFrameObjects Catch End Try End Sub +#End Region +#Region "Form functions" Friend Sub Focus(Optional ByVal Show As Boolean = False) If Not MF.Visible And Show Then MF.Show() If MF.Visible Then MF.BringToFront() : MF.Activate() @@ -45,6 +53,11 @@ Friend Class MainFrameObjects Dim a As Action = Sub() MF.BTT_TRAY_CLOSE_NO_SCRIPT.Visible = Settings.ClosingCommand.Attribute And Not Settings.ClosingCommand.IsEmptyString If MF.TRAY_CONTEXT.InvokeRequired Then MF.TRAY_CONTEXT.Invoke(a) Else a.Invoke End Sub + Friend Sub UpdateLogButton() + MyMainLOG_UpdateLogButton(MF.BTT_LOG, MF.Toolbar_TOP) + End Sub +#End Region +#Region "Notifications" Friend Overloads Sub ShowNotification(ByVal Message As String) MF.TrayIcon.ShowBalloonTip(2000, MF.TrayIcon.BalloonTipTitle, Message, ToolTipIcon.Info) End Sub @@ -63,4 +76,5 @@ Friend Class MainFrameObjects Focus() End If End Sub +#End Region End Class \ No newline at end of file diff --git a/SCrawler/MainMod.vb b/SCrawler/MainMod.vb index 7d49cf1..25f1bfd 100644 --- a/SCrawler/MainMod.vb +++ b/SCrawler/MainMod.vb @@ -136,18 +136,6 @@ Friend Module MainMod Return $"{If(Host?.Name, String.Empty)}{Opt}" End If End Function -#Region "Image Handlers management" - Friend Sub ImageHandler(ByVal User As IUserData) - ImageHandler(User, False) - ImageHandler(User, True) - End Sub - Friend Sub ImageHandler(ByVal User As IUserData, ByVal Add As Boolean) - MainFrameObj.ImageHandler(User, Add) - End Sub - Friend Sub CollectionHandler(ByVal [Collection] As UserDataBind) - MainFrameObj.CollectionHandler(Collection) - End Sub -#End Region #Region "Standalone video download functions" Friend Function GetCurrentBuffer() As String Dim b$ = BufferText diff --git a/SCrawler/My Project/AssemblyInfo.vb b/SCrawler/My Project/AssemblyInfo.vb index b73b108..ecba97d 100644 --- a/SCrawler/My Project/AssemblyInfo.vb +++ b/SCrawler/My Project/AssemblyInfo.vb @@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices ' by using the '*' as shown below: ' - - + + diff --git a/SCrawler/My Project/Resources.Designer.vb b/SCrawler/My Project/Resources.Designer.vb index 0565934..2e86318 100644 --- a/SCrawler/My Project/Resources.Designer.vb +++ b/SCrawler/My Project/Resources.Designer.vb @@ -260,6 +260,26 @@ Namespace My.Resources End Get End Property + ''' + ''' Looks up a localized resource of type System.Drawing.Icon similar to (Icon). + ''' + Friend ReadOnly Property RedGifsIcon() As System.Drawing.Icon + Get + Dim obj As Object = ResourceManager.GetObject("RedGifsIcon", resourceCulture) + Return CType(obj,System.Drawing.Icon) + End Get + End Property + + ''' + ''' Looks up a localized resource of type System.Drawing.Bitmap. + ''' + Friend ReadOnly Property RedGifsPic32() As System.Drawing.Bitmap + Get + Dim obj As Object = ResourceManager.GetObject("RedGifsPic32", resourceCulture) + Return CType(obj,System.Drawing.Bitmap) + End Get + End Property + ''' ''' Looks up a localized resource of type System.Drawing.Bitmap. ''' @@ -270,22 +290,12 @@ Namespace My.Resources End Get End Property - ''' - ''' Looks up a localized resource of type System.Drawing.Icon similar to (Icon). - ''' - Friend ReadOnly Property RegGifsIcon() As System.Drawing.Icon - Get - Dim obj As Object = ResourceManager.GetObject("RegGifsIcon", resourceCulture) - Return CType(obj,System.Drawing.Icon) - End Get - End Property - ''' ''' Looks up a localized resource of type System.Drawing.Bitmap. ''' - Friend ReadOnly Property RegGifsPic32() As System.Drawing.Bitmap + Friend ReadOnly Property RSSPic() As System.Drawing.Bitmap Get - Dim obj As Object = ResourceManager.GetObject("RegGifsPic32", resourceCulture) + Dim obj As Object = ResourceManager.GetObject("RSSPic", resourceCulture) Return CType(obj,System.Drawing.Bitmap) End Get End Property @@ -300,6 +310,16 @@ Namespace My.Resources End Get End Property + ''' + ''' Looks up a localized resource of type System.Drawing.Icon similar to (Icon). + ''' + Friend ReadOnly Property SettingsIcon() As System.Drawing.Icon + Get + Dim obj As Object = ResourceManager.GetObject("SettingsIcon", resourceCulture) + Return CType(obj,System.Drawing.Icon) + End Get + End Property + ''' ''' Looks up a localized resource of type System.Drawing.Bitmap. ''' diff --git a/SCrawler/My Project/Resources.resx b/SCrawler/My Project/Resources.resx index 714513d..be49678 100644 --- a/SCrawler/My Project/Resources.resx +++ b/SCrawler/My Project/Resources.resx @@ -199,10 +199,16 @@ ..\Content\Pictures\Pause_Blue_16.png;System.Drawing.Bitmap, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a - - ..\Content\Icons\RegGifsIcon.ico;System.Drawing.Icon, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a + + ..\Content\Icons\SettingsIcon.ico;System.Drawing.Icon, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a - - ..\Content\Pictures\RegGifsPic32.png;System.Drawing.Bitmap, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a + + ..\Content\Icons\RedGifsIcon.ico;System.Drawing.Icon, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a + + + ..\Content\Pictures\RedGifsPic32.png;System.Drawing.Bitmap, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a + + + ..\Content\Pictures\RSSPic.png;System.Drawing.Bitmap, System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a \ No newline at end of file diff --git a/SCrawler/PluginsEnvironment/Hosts/SettingsHost.vb b/SCrawler/PluginsEnvironment/Hosts/SettingsHost.vb index 597eae9..de767e0 100644 --- a/SCrawler/PluginsEnvironment/Hosts/SettingsHost.vb +++ b/SCrawler/PluginsEnvironment/Hosts/SettingsHost.vb @@ -79,10 +79,13 @@ Namespace Plugin.Hosts End Property Friend ReadOnly Property HasSpecialOptions As Boolean = False Private ReadOnly _ResponserGetMethod As MethodInfo + Private ReadOnly _ResponserIsContainer As Boolean = False Friend ReadOnly Property Responser As Response Get If Not _ResponserGetMethod Is Nothing Then Return _ResponserGetMethod.Invoke(Source, Nothing) + ElseIf _ResponserIsContainer Then + Return DirectCast(Source, IResponserContainer).Responser Else Return Nothing End If @@ -176,6 +179,7 @@ Namespace Plugin.Hosts Dim n() As String = {SettingsCLS.Name_Node_Sites, Name} If If(_XML(n)?.Count, 0) > 0 Then Source.Load(ToKeyValuePair(Of String, EContainer)(_XML(n))) Dim Members As IEnumerable(Of MemberInfo) = Plugin.GetType.GetTypeInfo.DeclaredMembers + _ResponserIsContainer = TypeOf Plugin Is IResponserContainer If Members.ListExists Then Dim Updaters As New List(Of MemberInfo) Dim Providers As New List(Of MemberInfo) @@ -307,6 +311,9 @@ Namespace Plugin.Hosts Throw New ArgumentNullException("IPluginContentProvider", $"Plugin [{Key}] does not provide user instance") End If End Function + Friend Function GetUserPostUrl(ByVal UserID As String, ByVal PostID As String) As String + Return Source.GetUserPostUrl(UserID, PostID) + End Function Private _AvailableValue As Boolean = True Private _AvailableAsked As Boolean = False Private _ActiveTaskCount As Integer = 0 diff --git a/SCrawler/PluginsEnvironment/Hosts/UserDataHost.vb b/SCrawler/PluginsEnvironment/Hosts/UserDataHost.vb index 9e6fa97..f4f2844 100644 --- a/SCrawler/PluginsEnvironment/Hosts/UserDataHost.vb +++ b/SCrawler/PluginsEnvironment/Hosts/UserDataHost.vb @@ -74,8 +74,6 @@ Namespace Plugin.Hosts UserSuspended = .UserSuspended End With End Sub - Protected Overrides Sub ReparseVideo(ByVal Token As CancellationToken) - End Sub Protected Overrides Sub DownloadContent(ByVal Token As CancellationToken) If UseInternalDownloader Then DownloadContentDefault(Token) @@ -105,7 +103,7 @@ Namespace Plugin.Hosts Progress.Perform(Count) End Sub Private Sub ExternalPlugin_TotalCountChanged(ByVal Count As Integer) - Progress.TotalCount += Count + Progress.Maximum += Count End Sub Protected Overrides Sub Dispose(ByVal disposing As Boolean) If disposing And Not disposedValue Then diff --git a/SCrawler/SCrawler.vbproj b/SCrawler/SCrawler.vbproj index 1e66ebf..a94e89e 100644 --- a/SCrawler/SCrawler.vbproj +++ b/SCrawler/SCrawler.vbproj @@ -169,6 +169,18 @@ Form + + DownloadFeedForm.vb + + + Form + + + FeedMedia.vb + + + UserControl + @@ -182,6 +194,12 @@ Form + + MissingPostsForm.vb + + + Form + SchedulerEditorForm.vb @@ -189,7 +207,13 @@ Form + + + True + True + Resources.resx + @@ -295,11 +319,6 @@ Application.myapp True - - True - True - Resources.resx - True Settings.settings @@ -344,12 +363,21 @@ ActiveDownloadingProgress.vb + + DownloadFeedForm.vb + DownloadSavedPostsForm.vb + + FeedMedia.vb + GroupEditorForm.vb + + MissingPostsForm.vb + SchedulerEditorForm.vb @@ -379,9 +407,9 @@ VbMyResourcesResXFileCodeGenerator - Resources.Designer.vb My.Resources Designer + Resources.Designer.vb VideosDownloaderForm.vb @@ -427,6 +455,11 @@ + + + + + PreserveNewest @@ -444,6 +477,33 @@ false - + + + {6BF52A50-394A-11D3-B153-00C04F79FAA6} + 1 + 0 + 0 + aximp + False + + + {00020430-0000-0000-C000-000000000046} + 2 + 0 + 0 + primary + False + True + + + {6BF52A50-394A-11D3-B153-00C04F79FAA6} + 1 + 0 + 0 + tlbimp + False + True + + \ No newline at end of file diff --git a/SCrawler/SettingsCLS.vb b/SCrawler/SettingsCLS.vb index 4a1b9e0..3d0c27a 100644 --- a/SCrawler/SettingsCLS.vb +++ b/SCrawler/SettingsCLS.vb @@ -18,6 +18,7 @@ Friend Class SettingsCLS : Implements IDisposable Friend Const DefaultMaxDownloadingTasks As Integer = 5 Friend Const Name_Node_Sites As String = "Sites" Private Const SitesValuesSeparator As String = "," + Friend Const CookieEncryptKey As String = "SCrawlerCookiesEncryptKeyword" Friend ReadOnly Design As XmlFile Private ReadOnly MyXML As XmlFile Private ReadOnly OS64 As Boolean @@ -54,6 +55,9 @@ Friend Class SettingsCLS : Implements IDisposable GlobalPath = New XMLValue(Of SFile)("GlobalPath", New SFile($"{SFile.GetPath(Application.StartupPath).PathWithSeparator}Data\"), MyXML,, New XMLValueBase.ToFilePath) + CookiesEncrypted = New XMLValue(Of Boolean)("CookiesEncrypted", False, MyXML) + EncryptCookies.CookiesEncrypted = CookiesEncrypted + SeparateVideoFolder = New XMLValue(Of Boolean)("SeparateVideoFolder", True, MyXML) CollectionsPath = New XMLValue(Of String)("CollectionsPath", "Collections", MyXML) @@ -68,6 +72,7 @@ Friend Class SettingsCLS : Implements IDisposable Dim tmpPluginList As IEnumerable(Of PluginHost) = PluginHost.GetPluginsHosts(MyXML, GlobalPath.Value, DefaultTemporary, DefaultDownloadImages, DefaultDownloadVideos) If tmpPluginList.ListExists Then Plugins.AddRange(tmpPluginList) + CookiesEncrypted.Value = True FastProfilesLoading = New XMLValue(Of Boolean)("FastProfilesLoading", False, MyXML) MaxLargeImageHeight = New XMLValue(Of Integer)("MaxLargeImageHeight", 150, MyXML) @@ -86,6 +91,9 @@ Friend Class SettingsCLS : Implements IDisposable ShowGroups = New XMLValue(Of Boolean)("ShowGroups", True, MyXML) UseGrouping = New XMLValue(Of Boolean)("UseGrouping", True, MyXML) + AddMissingToLog = New XMLValue(Of Boolean)("AddMissingToLog", True, MyXML) + AddMissingErrorsToLog = New XMLValue(Of Boolean)("AddMissingErrorsToLog", False, MyXML) + LatestSavingPath = New XMLValue(Of SFile)("LatestSavingPath", Nothing, MyXML,, New XMLValueBase.ToFilePath) LatestSelectedChannel = New XMLValue(Of String)("LatestSelectedChannel",, MyXML) LastUpdatedLimit = New XMLValue(Of Date) @@ -106,6 +114,11 @@ Friend Class SettingsCLS : Implements IDisposable ChannelsMaxJobsCount = New XMLValue(Of Integer)("MaxJobsCount", DefaultMaxDownloadingTasks, MyXML, n) ChannelsAddUserImagesFromAllChannels = New XMLValue(Of Boolean)("AddUserImagesFromAllChannels", True, MyXML, n) + n = {"Feed"} + FeedDataColumns = New XMLValue(Of Integer)("DataColumns", 1, MyXML, n) + FeedDataRows = New XMLValue(Of Integer)("DataRows", 10, MyXML, n) + FeedEndless = New XMLValue(Of Boolean)("Endless", True, MyXML, n) + n = {"Users"} FromChannelDownloadTop = New XMLValue(Of Integer)("FromChannelDownloadTop", 10, MyXML, n) FromChannelDownloadTopUse = New XMLValue(Of Boolean)("FromChannelDownloadTopUse", False, MyXML, n) @@ -204,7 +217,7 @@ Friend Class SettingsCLS : Implements IDisposable If d.Count > 0 Then For Each kv In d Users.Add(New UserDataBind(kv.Value(0).CollectionName)) - CollectionHandler(DirectCast(Users(Users.Count - 1), UserDataBind)) + MainFrameObj.CollectionHandler(DirectCast(Users(Users.Count - 1), UserDataBind)) For v = 0 To kv.Value.Count - 1 : DirectCast(Users(Users.Count - 1), UserDataBind).Add(kv.Value(v), False) : Next Next d.Clear() @@ -329,6 +342,53 @@ Friend Class SettingsCLS : Implements IDisposable _UserListUpdateRequired = True End Try End Sub + Friend Overloads Function GetUser(ByVal User As IUserData) As IUserData + If Users.Count > 0 Then + Dim uSimple As Predicate(Of IUserData) = Function(u) u.Equals(DirectCast(User, UserDataBase)) + Dim uCol As Predicate(Of IUserData) = Function(ByVal u As IUserData) As Boolean + If u.IsCollection Then + Return DirectCast(u, UserDataBind).Collections.Exists(uSimple) + Else + Return False + End If + End Function + Dim uu As Predicate(Of IUserData) + If User.IncludedInCollection Then uu = uCol Else uu = uSimple + Dim i% = Users.FindIndex(uu) + If i >= 0 Then + If Users(i).IsCollection Then + With DirectCast(Users(i), UserDataBind) + i = .Collections.FindIndex(uSimple) + If i >= 0 Then Return .Collections(i) + End With + Else + Return Users(i) + End If + End If + End If + Return Nothing + End Function + Friend Overloads Function GetUser(ByVal UserKey As String) As IUserData + If Users.Count > 0 Then + Dim finder As Predicate(Of IUserData) = Function(u) u.Key = UserKey + Dim i%, ii% + For i = 0 To Users.Count - 1 + With Users(i) + If .IsCollection Then + With DirectCast(.Self, UserDataBind) + If .Count > 0 Then + ii = .Collections.FindIndex(finder) + If ii >= 0 Then Return .Collections(ii) + End If + End With + Else + If finder.Invoke(.Self) Then Return .Self + End If + End With + Next + End If + Return Nothing + End Function #End Region Friend Sub UpdateBlackList() If BlackList.Count > 0 Then @@ -392,6 +452,9 @@ Friend Class SettingsCLS : Implements IDisposable End Property Friend ReadOnly Property MaxUsersJobsCount As XMLValue(Of Integer) Friend ReadOnly Property ImgurClientID As XMLValue(Of String) + Private ReadOnly Property CookiesEncrypted As XMLValue(Of Boolean) + Friend ReadOnly Property AddMissingToLog As XMLValue(Of Boolean) + Friend ReadOnly Property AddMissingErrorsToLog As XMLValue(Of Boolean) #Region "Defaults" Friend ReadOnly Property DefaultTemporary As XMLValue(Of Boolean) Friend ReadOnly Property DefaultDownloadImages As XMLValue(Of Boolean) @@ -459,6 +522,11 @@ Friend Class SettingsCLS : Implements IDisposable Friend ReadOnly Property ChannelsMaxJobsCount As XMLValue(Of Integer) Friend ReadOnly Property ChannelsAddUserImagesFromAllChannels As XMLValue(Of Boolean) #End Region +#Region "Feed properties" + Friend ReadOnly Property FeedDataColumns As XMLValue(Of Integer) + Friend ReadOnly Property FeedDataRows As XMLValue(Of Integer) + Friend ReadOnly Property FeedEndless As XMLValue(Of Boolean) +#End Region #Region "New version properties" Friend ReadOnly Property CheckUpdatesAtStart As XMLValue(Of Boolean) Friend ReadOnly Property ShowNewVersionNotification As XMLValue(Of Boolean)