geonu/SCrawler
1
0
Fork 0
mirror of https://github.com/AAndyProgram/SCrawler.git synced 2026-03-14 15:52:18 +00:00
Code Issues Packages Projects Releases Wiki Activity

2023.6.5.0

Browse Source 
YT settings: removed property 'ItemsListLimit', add property 'ReplaceModificationDate'
YT.MediaItem: fix 'Pending'
YT.VideoListForm: add 'Shift' to add without downloading; add 'F5' hot key to start download; remove list items limit; fix item 'Pending', fixed items queue

UserDataBase: add 'IconBannerDownloaded' properties; add 'HOST.Available' check to 'DownloadSingleObject'; update file deletion in 'DownloadContentDefault'; add truncating '_TempPostsList' if number of ids > 1000
Instagram: add authorization headers
Mastodon: implement 'DownloadIconBanner'; update 'ReparseMissing' function
Reddit: implement 'DownloadIconBanner'
Twitter: implement 'DownloadIconBanner'; update parsers to parse posts with two videos; implement gallery-dl for all function; remove headers from settings
Download.DownloadProgress: remove main progress perform when downloading saved posts
VideoDownloaderForm: bind the 'BTT_ADD_URLS_ARR' button to the 'BTT_ADD_KeyClick' function
UsersInfoForm: add folder opening on double click on an item
ListImagesLoader: fix refill bug when the number of filtered profiles = 0
TrayIcon: add standalone downloader to context menu
DownloadableMediaHost: fix a bug when not downloaded videos do not appear in the list when loading the program
This commit is contained in:
Andy
2023-06-05 19:36:35 +03:00
parent abdef81e5f
commit 938042ea9e
33 changed files with 938 additions and 635 deletions
Download Patch File Download Diff File Expand all files Collapse all files

19
Changelog.md
View File

@@ -1,3 +1,22 @@
# 2023.6.5.0
*2023-06-05*
- Added
- **Instagram**: add additional authorization headers
- Setting to prevent user icon and banner from downloading (Request #129)
- Add standalone downloader to tray context menu
- YouTube downloader: added `Replace modification date` property
- Minor improvements
- Fixed
- Fascist **Twitter**: posts not downloading (new API)
- Main window: refill bug when the number of filtered profiles = 0
- Standalone downloader: new items are not added to the queue
- Standalone downloader: bug when not downloaded videos do not appear in the list when loading the program
- Standalone downloader: add videos array not working
- Saved posts: remove main progress perform when downloading saved posts
- Minor bugs
# 2023.5.12.0 # 2023.5.12.0
*2023-05-12* *2023-05-12*

BIN
ProgramScreenshots/SettingsGlobalDownloading.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 18 KiB

After

Width:  |  Height:  |  Size: 19 KiB

BIN
ProgramScreenshots/SettingsSiteInstagram.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 27 KiB

After

Width:  |  Height:  |  Size: 37 KiB

BIN
ProgramScreenshots/SettingsSiteTwitter.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 19 KiB

After

Width:  |  Height:  |  Size: 17 KiB

6
SCrawler.YouTube/Base/YouTubeSettings.vb
View File

@@ -94,12 +94,12 @@ Namespace API.YouTube.Base
End Property End Property
#End Region #End Region
#Region "Defaults" #Region "Defaults"
<Browsable(True), GridVisible, XMLVN({"Defaults"}), Category("Defaults"), DisplayName("Replace modification date"),
Description("Set the file date to the date the video was added (website) (if available). Default: false.")>
Public ReadOnly Property ReplaceModificationDate As XMLValue(Of Boolean)
<Browsable(True), GridVisible, XMLVN({"Defaults"}), Category("Defaults"), DisplayName("Use cookies"), <Browsable(True), GridVisible, XMLVN({"Defaults"}), Category("Defaults"), DisplayName("Use cookies"),
Description("By default, use cookies when downloading from YouTube.")> Description("By default, use cookies when downloading from YouTube.")>
Public ReadOnly Property DefaultUseCookies As XMLValue(Of Boolean) Public ReadOnly Property DefaultUseCookies As XMLValue(Of Boolean)
<Browsable(True), GridVisible(False), XMLVN({"Defaults"}, 100), Category("Defaults"), DisplayName("Items limit"),
Description("Number of items displayed in the list.")>
Public ReadOnly Property ItemsListLimit As XMLValue(Of Integer)
<Browsable(True), GridVisible(False), XMLVN({"Defaults"}), Category("Defaults"), <Browsable(True), GridVisible(False), XMLVN({"Defaults"}), Category("Defaults"),
DisplayName("Auto remove"), Description("Automatically remove downloaded items from the list.")> DisplayName("Auto remove"), Description("Automatically remove downloaded items from the list.")>
Public ReadOnly Property RemoveDownloadedAutomatically As XMLValue(Of Boolean) Public ReadOnly Property RemoveDownloadedAutomatically As XMLValue(Of Boolean)

3
SCrawler.YouTube/Downloader/MediaItem.vb
View File

@@ -277,6 +277,7 @@ Namespace DownloadObjects.STDownloader
#Region "Context buttons' handlers" #Region "Context buttons' handlers"
Public Sub AddToQueue() Public Sub AddToQueue()
ControlInvokeFast(Me, Sub() ControlInvokeFast(Me, Sub()
Pending = True
BTT_DOWN.Visible = False BTT_DOWN.Visible = False
SEP_DOWN.Visible = False SEP_DOWN.Visible = False
End Sub, EDP.None) End Sub, EDP.None)
@@ -300,6 +301,8 @@ Namespace DownloadObjects.STDownloader
Throw oex Throw oex
Catch ex As Exception Catch ex As Exception
ErrorsDescriber.Execute(EDP.SendToLog, ex, $"MediaItem.Download:{vbCr}{MyContainer.ToString}{vbCr}{MyContainer.URL})") ErrorsDescriber.Execute(EDP.SendToLog, ex, $"MediaItem.Download:{vbCr}{MyContainer.ToString}{vbCr}{MyContainer.URL})")
Finally
Pending = False
End Try End Try
End Sub End Sub
#End Region #End Region

12
SCrawler.YouTube/Downloader/VideoListForm.Designer.vb generated
View File

@@ -143,7 +143,8 @@ Namespace DownloadObjects.STDownloader
Me.BTT_ADD.Size = New System.Drawing.Size(184, 22) Me.BTT_ADD.Size = New System.Drawing.Size(184, 22)
Me.BTT_ADD.Tag = "a" Me.BTT_ADD.Tag = "a"
Me.BTT_ADD.Text = "Add (Ins)" Me.BTT_ADD.Text = "Add (Ins)"
Me.BTT_ADD.ToolTipText = "Click to add." & Global.Microsoft.VisualBasic.ChrW(13) & Global.Microsoft.VisualBasic.ChrW(10) & "Ctrl+click to use cookies for download (if supported)." Me.BTT_ADD.ToolTipText = "Click to add." & Global.Microsoft.VisualBasic.ChrW(13) & Global.Microsoft.VisualBasic.ChrW(10) & "Ctrl+click to use cookies for download (if supported)." & Global.Microsoft.VisualBasic.ChrW(13) & Global.Microsoft.VisualBasic.ChrW(10) & "Shift to a" &
"dd without downloading."
' '
'BTT_ADD_PLS_ARR 'BTT_ADD_PLS_ARR
' '
@@ -154,7 +155,8 @@ Namespace DownloadObjects.STDownloader
Me.BTT_ADD_PLS_ARR.Size = New System.Drawing.Size(184, 22) Me.BTT_ADD_PLS_ARR.Size = New System.Drawing.Size(184, 22)
Me.BTT_ADD_PLS_ARR.Tag = "pls" Me.BTT_ADD_PLS_ARR.Tag = "pls"
Me.BTT_ADD_PLS_ARR.Text = "Add playlist array" Me.BTT_ADD_PLS_ARR.Text = "Add playlist array"
Me.BTT_ADD_PLS_ARR.ToolTipText = "Click to add." & Global.Microsoft.VisualBasic.ChrW(13) & Global.Microsoft.VisualBasic.ChrW(10) & "Ctrl+click to use cookies for download (if supported)." Me.BTT_ADD_PLS_ARR.ToolTipText = "Click to add." & Global.Microsoft.VisualBasic.ChrW(13) & Global.Microsoft.VisualBasic.ChrW(10) & "Ctrl+click to use cookies for download (if supported)." & Global.Microsoft.VisualBasic.ChrW(13) & Global.Microsoft.VisualBasic.ChrW(10) & "Shift to a" &
"dd without downloading."
' '
'BTT_ADD_NO_SHORTS 'BTT_ADD_NO_SHORTS
' '
@@ -166,7 +168,7 @@ Namespace DownloadObjects.STDownloader
Me.BTT_ADD_NO_SHORTS.Tag = "ans" Me.BTT_ADD_NO_SHORTS.Tag = "ans"
Me.BTT_ADD_NO_SHORTS.Text = "Add (without Shorts)" Me.BTT_ADD_NO_SHORTS.Text = "Add (without Shorts)"
Me.BTT_ADD_NO_SHORTS.ToolTipText = "Download all videos except 'Shorts'." & Global.Microsoft.VisualBasic.ChrW(13) & Global.Microsoft.VisualBasic.ChrW(10) & "Click to add." & Global.Microsoft.VisualBasic.ChrW(13) & Global.Microsoft.VisualBasic.ChrW(10) & "Ctrl+click to use cookies fo" & Me.BTT_ADD_NO_SHORTS.ToolTipText = "Download all videos except 'Shorts'." & Global.Microsoft.VisualBasic.ChrW(13) & Global.Microsoft.VisualBasic.ChrW(10) & "Click to add." & Global.Microsoft.VisualBasic.ChrW(13) & Global.Microsoft.VisualBasic.ChrW(10) & "Ctrl+click to use cookies fo" &
"r download (if supported)." "r download (if supported)." & Global.Microsoft.VisualBasic.ChrW(13) & Global.Microsoft.VisualBasic.ChrW(10) & "Shift to add without downloading."
' '
'BTT_ADD_SHORTS_ONLY 'BTT_ADD_SHORTS_ONLY
' '
@@ -178,7 +180,7 @@ Namespace DownloadObjects.STDownloader
Me.BTT_ADD_SHORTS_ONLY.Tag = "as" Me.BTT_ADD_SHORTS_ONLY.Tag = "as"
Me.BTT_ADD_SHORTS_ONLY.Text = "Add (Shorts only)" Me.BTT_ADD_SHORTS_ONLY.Text = "Add (Shorts only)"
Me.BTT_ADD_SHORTS_ONLY.ToolTipText = "Download only 'Shorts' videos." & Global.Microsoft.VisualBasic.ChrW(13) & Global.Microsoft.VisualBasic.ChrW(10) & "Click to add." & Global.Microsoft.VisualBasic.ChrW(13) & Global.Microsoft.VisualBasic.ChrW(10) & "Ctrl+click to use cookies for down" & Me.BTT_ADD_SHORTS_ONLY.ToolTipText = "Download only 'Shorts' videos." & Global.Microsoft.VisualBasic.ChrW(13) & Global.Microsoft.VisualBasic.ChrW(10) & "Click to add." & Global.Microsoft.VisualBasic.ChrW(13) & Global.Microsoft.VisualBasic.ChrW(10) & "Ctrl+click to use cookies for down" &
"load (if supported)." "load (if supported)." & Global.Microsoft.VisualBasic.ChrW(13) & Global.Microsoft.VisualBasic.ChrW(10) & "Shift to add without downloading."
' '
'BTT_DOWN 'BTT_DOWN
' '
@@ -187,7 +189,7 @@ Namespace DownloadObjects.STDownloader
Me.BTT_DOWN.Name = "BTT_DOWN" Me.BTT_DOWN.Name = "BTT_DOWN"
Me.BTT_DOWN.Size = New System.Drawing.Size(81, 22) Me.BTT_DOWN.Size = New System.Drawing.Size(81, 22)
Me.BTT_DOWN.Text = "Download" Me.BTT_DOWN.Text = "Download"
Me.BTT_DOWN.ToolTipText = "Download pending items" Me.BTT_DOWN.ToolTipText = "Download pending items (F5)"
' '
'BTT_STOP 'BTT_STOP
' '

330
SCrawler.YouTube/Downloader/VideoListForm.resx
View File

@@ -134,6 +134,27 @@
</metadata> </metadata>
<assembly alias="System.Drawing" name="System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" /> <assembly alias="System.Drawing" name="System.Drawing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
<data name="BTT_ADD.Image" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64"> <data name="BTT_ADD.Image" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
<value>
iVBORw0KGgoAAAANSUhEUgAAABgAAAAYCAYAAADgdz34AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8
YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAAN1SURBVEhLrZVJTFNRFIafQhgD1OBUpiiKYg22AopFKggK
FdRYQUEZgsqgGBAChaiYRlG2RuPOuCDGGDcG48phgcQog0KFV4QKKZ2wSIJxf83vuc8SWRAw8E7yp23u
yf/de95/X4Wlyu+eT4f/fR8sJL7mbVt+cSOVWcUyB3U4/EUHvUiy6HDKliVBvG3LL26SMKRiQd0CQj8K
CO8XEPnZF5WuXPkAGlHFwnoErOtfhU2D/vQ7FM2efPkASQTgO481+yNxJAwZoxtwe6ZEPoCWALHmAOwZ
USDHGkHz34p7s1XyAc6LGpZsUSDXGomzk9tRZdfg0a86+QCN4j6m/xaFUvsO1DgS0ehMxZNfxv8DLJbz
OZX2p/8uscej1pGMlikdbnqy0fGzCWufBi4q5fPgDmmHWXMZp3znjepw0pqJMlsOap3HcdV9Cnc8ZfQ9
GdfcB3DLk4O2aT06ZpvweNYogR7O1uHuTAVuTRej2V1AI8xDjb0Qys5gSABuHvqBMt4rIGrQD2qKIU+K
YSIW5+w7ccWZgutTB8lAj/bpXAlw83s2melQ5dDg9EQcssY2Qm0JRfSAHxTkU2gx/APwnXNzvqi2hFCz
Evle83pXClrdmWjz6AmQjRvfM2F0p9GzSEL5pAonxjcjfXQ9EsQQRAz6QtEnIJAuZf6XeYC8ER1iyFxD
O8iyKqUYVjrUaHDtxbWpDLSS6VV3OoyuNNQ796Ca1krtKhwb34QDdFLJfMAX/DJy81WvBRj65gHOWA9B
OxJOMYxCsS0edfQwmygpLWTITRtdqZIaaFTVNBKepqPjMUj7uh47hoOh/OSDEBpxQJcA4RWpkwDv5wEu
iYnMKKYy02gpM41dYCZrNTN9q6LPi8wkXmYPflwi872otKtRNLkNuePRaHGWIb8n43eFuJ+ViFpWREol
aYdIn0m9WiYBeEwXiticeFO72MIqHbtQZIvDETplCp22xlEkrS0qHtOlijeaxAZ2mp5LzlgkkuhGbzEH
otx2UjLxti2//gJqWPZYhPSi42/TcErKWat3xiutv4AKttsShihKyhqKM/9vKJjL+UqLm9SJxSx6wB8K
imHQOwE+byglZhkB5ylh3DyAdr6ax/AlAXpkBBRT/AJp5wJdIOEF6RkBumUEFAwbpJEYekl0gQxdpLdy
ASjL3GhBLZlzQfgD9Y2tq0N6ki0AAAAASUVORK5CYII=
</value>
</data>
<data name="BTT_ADD_PLS_ARR.Image" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
<value> <value>
iVBORw0KGgoAAAANSUhEUgAAABgAAAAYCAYAAADgdz34AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8 iVBORw0KGgoAAAANSUhEUgAAABgAAAAYCAYAAADgdz34AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8
YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAAN5SURBVEhLrZVJTFNRFIafljBIgBqcymAU54oWGSRCK1oU YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAAN5SURBVEhLrZVJTFNRFIafljBIgBqcymAU54oWGSRCK1oU
@@ -152,206 +173,185 @@
KksYKp0HJZPAttkXNzGLdSxnIEZ60fG3aTQl5Yg9MOO51l9AFdtijUIcJWUhxZl/Gwoncj7X4iYNYimL KksYKp0HJZPAttkXNzGLdSxnIEZ60fG3aTQl5Yg9MOO51l9AFdtijUIcJWUhxZl/Gwoncj7X4iYNYimL
7w2BkmK44LUAxQtKiUVGwHFRy7h5KHU+n8fwKQE+yAgopfiFUecCXSDhCekhAd7ICCj8apRGYuwi0QUy 7w2BkmK44LUAxQtKiUVGwHFRy7h5KHU+n8fwKQE+yAgopfiFUecCXSDhCekhAd7ICCj8apRGYuwi0QUy
viK9lAtAWeZGU2rGnAvCHy5drfKWDYjrAAAAAElFTkSuQmCC viK9lAtAWeZGU2rGnAvCHy5drfKWDYjrAAAAAElFTkSuQmCC
</value>
</data>
<data name="BTT_ADD_PLS_ARR.Image" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
<value>
iVBORw0KGgoAAAANSUhEUgAAABgAAAAYCAYAAADgdz34AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8
YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAAN7SURBVEhLrZVJTFNRFIafQhgkQA1OZYriSI0UtUI0vIKg
UEGNBRSUIQ4MihElUAgOqeKwcGM07owLYoxxYzSuHBZIjAoKrfpaoBZLJyiaYNxf83vus0QWBAy+k/xp
k3vzf+ee99/3hNkq7GZIZ/itEEwnvhbcNvfiRmlWDcu1iNj5UYRBItlE7HflyZDgtrkXN9n4ScMWdAuI
eSsg7r2AhL5Q1PoKlQPoJA2LfSdg8ft5WG4JR7oUg9ZAiXKATALwzlOs4dhkj0XO4FJc+V6pHECU0liK
NQJb7CoUOOJp/qtwc6JOOcBRScd0NhUKHQk4NLIWde503P3ZqBygWcpihi+JqHKnosGzCc3ebbj/0/Rv
gJlyPqkjluxfle51OOXRoW1UxKVAPjp/tGDRg8gZpX4U1Sl3mDeZccp30aCIYkcuql0FOOXdi3b/flwN
VNN/Hc769egIFODyuAGdEy24N2GSQXcmGnHjew06xivQ6i+lERahwV0G9eMoyABuHvOGMt4jINESBi3F
kCfFOJyCI+71OO3NxLnR7WRgwLXxQhlwaSyfzETUedJxYHg18oaWQWuLQVJ/GFTkU2Yz/gXwzrk5X9Ta
ommzGiVB8zO+TJz35+JywECAfFwYy4XJn0XPYjMOj2iwz7kC2YNLsEGKRrwlFKpeAZF0KUs+TgEU2UUk
k3k6dZDnUMsxrPVo0eTLwNnRHJwn03Z/Nky+LJzxbkE9rVW5NdjjXA49nVQ27w8Fv4zcfN5zAcbeKYCD
jh3Yao+jGCaiwrUOjfQwWygpbWTITZt922Q10ajqaSQ8TbudycgaWILUz1FQfwhBNI04okuA8Iz0mACv
pwBOSBnMNKBnZmcVMw8fY+av9czsqqPf4+yi/SS7/e0EmWeg1q1F+cgaFDqT0OatRtmHHb9qpGxWKYms
nKQniZ9IfaQekckAHtPpIjYpvun6QBur9aSh3LUau+iUmXTaBk+5vDajeExnK76xw97EDtBzKRhKwGa6
0SutkTjsKpZNgtvmXtzELDWw/KF4+UXH36ZxlJRDjuCM/7f+AGrYRlssEikpCynO/NtQOpnz/y1u0ihV
sKT+cKgohgteCQh5QSmxKgg4KukZN4+gzufzGD4lwDsFARUUv0jqXKALJDwhPSRAt4KA0s9GeSTGHhJd
IGMX6aVSAMoyN5pWs+ZcEH4DgcGuQfDpaFIAAAAASUVORK5CYII=
</value> </value>
</data> </data>
<data name="BTT_ADD_NO_SHORTS.Image" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64"> <data name="BTT_ADD_NO_SHORTS.Image" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
<value> <value>
iVBORw0KGgoAAAANSUhEUgAAABgAAAAYCAYAAADgdz34AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8 iVBORw0KGgoAAAANSUhEUgAAABgAAAAYCAYAAADgdz34AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8
YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAAOBSURBVEhLrZVZSFRRGMdvKS6JOmHbuETZpkaN1ZQYTtZY YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAAN9SURBVEhLrZVZSFRRGMdvKa6oE5Y1bpTtRY1N5pB407Tu
zp2xbVLLcqHMpaws0UnaqMx6iYiinqIHiYheouip5aEkKjV1qhmXmzXeWXSmwOj9xL/vXEbyQTTsfvCH TFrRpJXlgpVLTWiJTlIZli0QPUTLW/QgEdFLFD21PFREZYtONTON08jsOiYYvp/4953LSD6Iht0P/jBw
gXP4/77znf+5I0xWYTdCWsJvhmA88bXgtqkXN8q065mx24AtHw0QHSSnAYWuHAUS3Db14iZZn/RsRquA Dv/fd77zP3eE6SrqekR39I0ITCa+Ft428+JGOouWFfWJ2PpFhMFKsonY4y6WIeFtMy9ukvdVy+JeC0h8
mHcC4joEJHSGotJrUQ+Q7dCz2PcCZndMw4LucKQ7YnDSn68ewEgA3nmyPRyre2KxsW8umn+UqAcQHZks JyD5o4C0z5GoD5QoBxCtWpb0XsC8j7OwsC8a2dZEnAyVKQcoJADvPMsSDa09CYWO+bg4UqUcQLLqWJYl
2R6BtT0amKR4mv9i3BipUg9Q7sxmeqcGFikB+waXoUpOx91fteoB6iUTE78kolRORY17Neo963H/l+3f BhvsKuidqTT/Jbg+2qAc4JBVZDk2FUqcaTjgWY4GbzbujDUrB2h1bGWGH+mo9q6EyadFqz8P98bM/waY
ABPlfFRHJfPvEjkFx9x6NA4ZcNGfi5afDZj1IHJCaR9FtSgd5oxmnPKd12fALsmIMpcJxzzbccpXiMv+ KufjMjn0v6u8K9Dky0H7oIjzIQndv9ow937slFI/jO+WOywezzjlu9QhYrezCDVuPZr8O3EquAeXQjX0
Mvqtx2nfBjT5TbgUENEy0oB7IzYFdGekFtd/VKApUIyTvgIaYR5q5D3QPo6CAuDmMW8p420CErvDoKMY Oweng5vQFdLjwrAB3aNtuDtqlkG3R5txbaQOXcOVOBkspxGWwuTdB/WjeMgAbp74ljLeIyC9LwoaiiFP
8qRYvybjgLwcxz0ZODO0iQxEXAlYFMDF4VwyM6DKnY7dX5cgp38edM4YJHWFQUM+e5zWvwDeOTfnizpn inEgCwe9q3Hcr8OZwc1kYMDl4RIZcH5IIjMRDb5s7B1YiuL+BdDYEpHRGwUV+eyzGf8CeOfcnC9qbAm0
NG3WIj9ofsKbgbM+Iy75RQLk4tywETZfFt3FGuwfTMPOgYXI7puDFY5oxHeHQtMuIJIeZf7HMYC8HgPm WY2ysPmJgA4dwSJcCBkIIOHsUBHMwXy6i/Wo9azCLtciFDhSsMaagNS+SKg+CIilR1n2ZQKg1C4ik8yz
k3k6dZAjaZUYVrp1qPOuw+mhjThLpqd82bB5s3DCsxbVtFYqp2HbwAJsoJMq5l2h4I+Rm097LsDaPgaw qYNip1qOYb1Pg5ZALk4PFqKDTE8FC2AO5OOEfwMaaa3auwo7XAuxiU4qm/dGgj9Gbj7rmQDjhwmA/c4t
V9qMzJ44imEiil0pqKXLbKCkNJIhN633rldUR6OqppHwNG0dmI+s3jlI/RwF7YcQRNOII14JEJ6RHhPg 2GhPphimo9K9As10mW2UlHYy5KatgTxZLTSqRhoJT9N2Vybyv6dg5bd4qD9FIIFGHPNSgPCU9IgAbyYA
zRjAYdcmZnOb2fnhUnbBf5A1B6pZ8/cqdiVwiF2Vj7Bb3w+T+TpUyjoUDS6FZSAJjZ4ylPfu+F0hWVhJ jg4UMLNHzzqD1axz8DDrGmpkXaEGdnHoCLviPsZu/TxK5rmo92pQ4VmGElcG2v01qLXt+F3nMLAqm8Qq
r8iKHCIzk8RPpE5Sm8gUAI/peBEbFd90293IKt0rUeRaAjOdMoNOW+MuUtYmFI/pZMU3XpPr2G66F1N/ rBLTk6SvpM+kHonJAB7TySI2Lr7ppqed1fvWosK9FNvolDo6rclXIa9NKR7T6YpvvOpuYXvpXvT9aVhP
AtbQi15kj8R+1y7FJLht6sVNmr7VsNz+eOVDx7+mcZSUfVJwxv9b3OT8lwq2yhmLRErKTIoz/28oGM35 L3qxJRa17t2ySXjbzIubnHOZmNSfKn/o+Nc0mZJywBme8f8WN+nsr2PrbElIp6TMoTjz/4by8Zz/b3GT
/xY3qe0rZkld4dBQDGe8FhDyglJiVxFQ7jAzbh5BnU/nMXxKgPcqAoopfpHUuUAPSHhCekiAVhUBBZ+t Znsly+iNhopiGPdKQMRzSolFQcAhq55x8xjqfDaP4RMCvFcQUEnxi6XOBXpAwmPSAwK8VhBQ/s0oj8TY
ykisbSR6QNZXpJdqASjL3GhcTZpzQfgDSh6wcB+AKZwAAAAASUVORK5CYII= Q6IHZHxJeqEUgLLMjSbVtDkXhD9St6/+w21JdAAAAABJRU5ErkJggg==
</value> </value>
</data> </data>
<data name="BTT_ADD_SHORTS_ONLY.Image" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64"> <data name="BTT_ADD_SHORTS_ONLY.Image" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
<value> <value>
iVBORw0KGgoAAAANSUhEUgAAABgAAAAYCAYAAADgdz34AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8 iVBORw0KGgoAAAANSUhEUgAAABgAAAAYCAYAAADgdz34AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8
YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAAOBSURBVEhLrZVZSFRRGMdvKS6JOmHbuETZpkaN1ZQYTtZY YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAAN9SURBVEhLrZVZSFRRGMdvKa6oE5Y1bpTtRY1N5pB407Tu
zp2xbVLLcqHMpaws0UnaqMx6iYiinqIHiYheouip5aEkKjV1qhmXmzXeWXSmwOj9xL/vXEbyQTTsfvCH TFrRpJXlgpVLTWiJTlIZli0QPUTLW/QgEdFLFD21PFREZYtONTON08jsOiYYvp/4953LSD6Iht0P/jBw
gXP4/77znf+5I0xWYTdCWsJvhmA88bXgtqkXN8q065mx24AtHw0QHSSnAYWuHAUS3Db14iZZn/RsRquA Dv/fd77zP3eE6SrqekR39I0ITCa+Ft428+JGOouWFfWJ2PpFhMFKsonY4y6WIeFtMy9ukvdVy+JeC0h8
mHcC4joEJHSGotJrUQ+Q7dCz2PcCZndMw4LucKQ7YnDSn68ewEgA3nmyPRyre2KxsW8umn+UqAcQHZks JyD5o4C0z5GoD5QoBxCtWpb0XsC8j7OwsC8a2dZEnAyVKQcoJADvPMsSDa09CYWO+bg4UqUcQLLqWJYl
2R6BtT0amKR4mv9i3BipUg9Q7sxmeqcGFikB+waXoUpOx91fteoB6iUTE78kolRORY17Neo963H/l+3f BhvsKuidqTT/Jbg+2qAc4JBVZDk2FUqcaTjgWY4GbzbujDUrB2h1bGWGH+mo9q6EyadFqz8P98bM/waY
ABPlfFRHJfPvEjkFx9x6NA4ZcNGfi5afDZj1IHJCaR9FtSgd5oxmnPKd12fALsmIMpcJxzzbccpXiMv+ KufjMjn0v6u8K9Dky0H7oIjzIQndv9ow937slFI/jO+WOywezzjlu9QhYrezCDVuPZr8O3EquAeXQjX0
Mvqtx2nfBjT5TbgUENEy0oB7IzYFdGekFtd/VKApUIyTvgIaYR5q5D3QPo6CAuDmMW8p420CErvDoKMY Oweng5vQFdLjwrAB3aNtuDtqlkG3R5txbaQOXcOVOBkspxGWwuTdB/WjeMgAbp74ljLeIyC9LwoaiiFP
8qRYvybjgLwcxz0ZODO0iQxEXAlYFMDF4VwyM6DKnY7dX5cgp38edM4YJHWFQUM+e5zWvwDeOTfnizpn inEgCwe9q3Hcr8OZwc1kYMDl4RIZcH5IIjMRDb5s7B1YiuL+BdDYEpHRGwUV+eyzGf8CeOfcnC9qbAm0
NG3WIj9ofsKbgbM+Iy75RQLk4tywETZfFt3FGuwfTMPOgYXI7puDFY5oxHeHQtMuIJIeZf7HMYC8HgPm WY2ysPmJgA4dwSJcCBkIIOHsUBHMwXy6i/Wo9azCLtciFDhSsMaagNS+SKg+CIilR1n2ZQKg1C4ik8yz
k3k6dZAjaZUYVrp1qPOuw+mhjThLpqd82bB5s3DCsxbVtFYqp2HbwAJsoJMq5l2h4I+Rm097LsDaPgaw qYNip1qOYb1Pg5ZALk4PFqKDTE8FC2AO5OOEfwMaaa3auwo7XAuxiU4qm/dGgj9Gbj7rmQDjhwmA/c4t
V9qMzJ44imEiil0pqKXLbKCkNJIhN633rldUR6OqppHwNG0dmI+s3jlI/RwF7YcQRNOII14JEJ6RHhPg 2GhPphimo9K9As10mW2UlHYy5KatgTxZLTSqRhoJT9N2Vybyv6dg5bd4qD9FIIFGHPNSgPCU9IgAbyYA
zRjAYdcmZnOb2fnhUnbBf5A1B6pZ8/cqdiVwiF2Vj7Bb3w+T+TpUyjoUDS6FZSAJjZ4ylPfu+F0hWVhJ jg4UMLNHzzqD1axz8DDrGmpkXaEGdnHoCLviPsZu/TxK5rmo92pQ4VmGElcG2v01qLXt+F3nMLAqm8Qq
r8iKHCIzk8RPpE5Sm8gUAI/peBEbFd90293IKt0rUeRaAjOdMoNOW+MuUtYmFI/pZMU3XpPr2G66F1N/ rBLTk6SvpM+kHonJAB7TySI2Lr7ppqed1fvWosK9FNvolDo6rclXIa9NKR7T6YpvvOpuYXvpXvT9aVhP
AtbQi15kj8R+1y7FJLht6sVNmr7VsNz+eOVDx7+mcZSUfVJwxv9b3OT8lwq2yhmLRErKTIoz/28oGM35 L3qxJRa17t2ySXjbzIubnHOZmNSfKn/o+Nc0mZJywBme8f8WN+nsr2PrbElIp6TMoTjz/4by8Zz/b3GT
/xY3qe0rZkld4dBQDGe8FhDyglJiVxFQ7jAzbh5BnU/nMXxKgPcqAoopfpHUuUAPSHhCekiAVhUBBZ+t Znsly+iNhopiGPdKQMRzSolFQcAhq55x8xjqfDaP4RMCvFcQUEnxi6XOBXpAwmPSAwK8VhBQ/s0oj8TY
ykisbSR6QNZXpJdqASjL3GhcTZpzQfgDSh6wcB+AKZwAAAAASUVORK5CYII= Q6IHZHxJeqEUgLLMjSbVtDkXhD9St6/+w21JdAAAAABJRU5ErkJggg==
</value> </value>
</data> </data>
<data name="MENU_ADD.Image" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64"> <data name="MENU_ADD.Image" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
<value> <value>
iVBORw0KGgoAAAANSUhEUgAAABgAAAAYCAYAAADgdz34AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8 iVBORw0KGgoAAAANSUhEUgAAABgAAAAYCAYAAADgdz34AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8
YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAAOASURBVEhLrZVZSFRRGMdvKS6JOmHbuGG2FzXWaLY4Zlpz YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAAN8SURBVEhLrZVZSJRRGIZ/U1wSdcK2caMsW6nR3Chm1LRZ
b1rRpJblQotLKVmik7QhZvUQQRS+RQ8SEb2E0VPLQ0VUtuhkM5MzaTN3NscCo/cT/75zGckH0bD7wR8G UosmTS2XNpc0tEQnaUPMiugmiu6iC4mIbsLoquXCJCpbdMx/XKapcVbHAqP7E2/f+RvJC9Gw/4MXBs7h
zuH/+853/ueOMFVF3AjrirwZhonE10Lbpl/caINFz/L7DNj+yQDJSrIZUOoqUCChbdMvbrK5X89mvRQQ fb7znff8I8xXwTcDu0JuBWI28TX/toUXN8owp7DcAQ20gxoYRJJFgwP2PAni37bw4ibbP6Wwxb0CIt8I
90ZAwnsBSR/DUeMrVA+Qa9Wz+LcC5r6fgbS+SGRY43A6WKweYCsBeOfplkiss8cjb2A+Lv2oUA8gWjew iH4vIPZjEGrc+fIB1GIKi3orYNn7AKwaCEGyGIkzviL5ANkE4J0nmkOwbTgKOaMrcPl7hXwArZjBEs2h
dEsUsuwaiM5Emv9i3BitVQ9wxJrLMm0aFDqTcNC9DLVyBm7/alQP0OwwMulrMirlFaj3rEOzdxPu/jL/ SB9WQG+Nofmvxc2pWvkAx0Q1S7MokG+NxaHx9ah1JOPuzyb5AC0ju5jhcxwqHRvR4NyGFtcO3P9p+jfA
G2CynI+pwSH9rpCX44QnE60BA9qDRnT9bMGce9GTSvsgpkvpsGAs45TvogED9jrzUeUSccK7G2f8pbgc XDmfVv2I7leFYwManWlo82rQ4dOh60crlj4Im1PKR+FdUod50xmnfBeMarDfmosqux6Nrr046zmAK74q
rKLfmTjrz8XFoIiOEQldoy24M2pWQLdGG3H9RzUujpTjtL+ERliEenk/tN0xUADcPO41ZbxHQHJfBHQU +p2Gc54sXPLp0TlpQNdUK+5NmSTQnakm3PhejUuT5TjjKaYRFqDBUQpldzj+XCKZR76mjPcJiBsIhopi
Q54U01A6DsurcNKbjXOBrWQg4cpIoQJoHzaSmQG1ngzsG1qCAscC6GxxSOmNgIZ89ttMfwG8c27OF3W2 yJNi/JKIo47NOOXKxHnvTjIw4OpkvgTomNCRmQa1zmSUfElC3thKqCyRiO8PhoJ8Si3GvwDeOTfniypL
WNqsRXHI/JQvG+f9+egISgQw4sJwPsz+HLoLPQ65V2LP4EJsGZiH1dZYJPaFQ/NOQDQ9yuJP4wBFdgNS BG1WoshvftqdiQueXHT6DATQ4eJELkweNd1FKo6Mb8I+22pkjy7HFjECMQNBULwTEEaPsmhwBqBgWIME
yTyDOihwapUY1nh0aPKtx9lAHs6T6Rn/Fph9OTjlzUIdrVXKK7FrMA25dFLFvDcc/DFy8xlPBJjejQMc Mk+mDvKsSimGNU4Vmt0ZOOfNwQUyPevJhsmtxmlXOupordKxCXtsq5BFJ5XM+4PAHyM3D3gmwPhuBuCg
cG7DRnsCxTAZ5a7laKTLbKGktJIhN232bVLURKOqo5HwNO0cTEXOl3lY8TkG2g9hiKURRz0XIDwmdRPg dRe2D0dTDONQbt+AJrrMVkpKGxly0xb3DknNNKo6GglPU6EtAeqR5dg4FA7lh0BE0IhDewQIT0ndBHg1
1TjA8W95zCxLrC1QydqGj7KOYB3rGKlll4PH2FV3A+v8fpzM16NG1qHMvRSFgylo9VbhsH3372rHDlZh A1Bvy2Imu461uytZu+c46/DWsY6JWtbpPcGufT3Jbn+rJ/MM1DhUKBtfh3xbPNpcVTgsFv6qHtGzClHL
F1mZVWQSSewnfST1iEwB8JhOFLEx8U2dciur8axBmWsJdtAps+m09Z4yZW1S8ZhOVXzjNXcT20f3IjqS ykg6kvYT6SOpT8skAI/pbBGbFt90y97GapxbUWZPwm46ZSadtsFZJq3NKR7T+YpvvP61mZXQvejHYpFK
oKcXvcgSjUOuvYpJaNv0i5u0D9UzoyNR+dDxr2kCJeWgMzTj/y1u0uasZmtt8UimpMymOPP/hpKxnP9v L3qNOQxH7PslE/+2hRc3af/cwHRjMdKHjn9Noykph6z+Gf9vSYDRapZiiUIcJWUJxZn/NxRP5/x/i5s0
cZPGL+UspTcSGorhrBcCwp5SSiwqAo5YJcbNo6jzmTyGjwjwVkVAOcUvmjoX6AEJD0n3CfBSRUDJZ5My WcpZfH8IFBTDxS8FBD6nlJhlBBwTdYybh1Lni3gMnxDgrYyAcopfGHUu0AMSHpMeEqBXRkDxkFEaibGP
ElMPiR6Q6TnpmVoAyjI3mlBT5lwQ/gBJOLA2lU2mdgAAAABJRU5ErkJggg== RA/I2EN6IReAssyNZtW8OReE31w2r8aW2OYjAAAAAElFTkSuQmCC
</value> </value>
</data> </data>
<data name="BTT_STOP.Image" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64"> <data name="BTT_STOP.Image" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
<value> <value>
iVBORw0KGgoAAAANSUhEUgAAABgAAAAYCAYAAADgdz34AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8 iVBORw0KGgoAAAANSUhEUgAAABgAAAAYCAYAAADgdz34AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8
YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAAVESURBVEhLjZVtTFNXGMcLQmdHO6AdarLCHOIAgRZ6a2nd YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAAVFSURBVEhLjZVtTFNXGMcLQmdHO6AdarLSOcQBQgv01tK6
FBwvUrRgSwUdI2pGxRUUX1CiGI3RRKOYLNmH7cP2YctMZtwSXZaNCaNmTiBDBm7iKkWHtZS+Fy57SRbo oSgvUrRgSwUdI2pGxRUUX1CiGI3RRONLsmQftg/bhy0zmXFLdFk2JkjJmMCmCE4wvOmwllL6Cpe9JAv0
2f+UVmcsG0/yy+095zn/59/nnHsvh8ZQSUmcKTX1w+vLllmupaR8sFEsfgnDMcHJ/4kRhUL5W0rKjdGl 7H9KqzOWjSf55fae85z/8+9zzr2XQ6M3Pz/KnJDwSduyZcM3JJKPN4nFr2E4wj/5PzGkVKp+k0jaR5Yu
SwevpKXpMRQHooKTNO4Yjfz+kpLPR1pbydiFC2SopSVwTanszhcIVmI6di4rcpgZ5nX7tm0P2XPnyHR7 7bmWmGjAUBQI80/S6DOZ+Lfz878aamggo5cukbv19b4bKlVrlkCwEtORc1mhY4Bh3rZt3/6YPX+eTF+8
O7FUV3vfy8hoTImPT8T0XJHuFSvev3fgAHlw8iQZO3GC2JBsPXyYdCiVNyQCwatIiVhkVKV6A+JW9vx5 SIbLytwfJifXSKKjYzE9V6R1xYqPBg8eJI9OnSKjJ08SK5ItR46QJpWqXSYQvImUkEVG1OoNELewFy4Q
wmIt29ZGpo8fJ/c0Gt+XYvG7SBGCGM51kchGxR+0tJCHR48SG5Imzpwhd1tbA9cUih9yBIJ0JD5VBG0p FmvZxkYyfeIEGdRqPd+IxR8gRQgiOG0ikZWKP6qvJ4+PHSNWJI2fPUv6Ghp8N5TKn6QCQRISnyuCthTY
tdfU2KhzKj4NcfbQITINY/66OmJKShpGmhIIOF3JyRct9fVkDAWswHbkCLFjgfP0aWJGu7qUyt5sPj8D ysut1DkVn4Y4e/gwmYYxb2UlMcfF9SNNBQSclvj4y8NVVWQUBSzAevQosWHBxJkz5AHa1aJSdabx+clI
ycEiIzJZ6UR1tW3q7FnCwsw0TLFYN71/P5ncvZt0KBTu7SLRZaSWAyFHt3z5yx1y+Y3BnTsDj5A0Dibg 9hcZkssLxsvKrFPnzhEWZqZhisW66QMHyOSePaRJqXTuEImuIrUICDn65ctfb1Io2rt37fI9QdIYGIeb
xok98aBlroMHyXUUyePzM39imJJxvd4+depUsCUsHLNoL9vcTKYgbsrL86bzeJ9A2AAkgG44J5YRiTJQ CeyJCy1zHDpE2lAkk89Pucsw+WMGg23q9Gl/S1g4ZtFetq6OTEHcnJnpTuLxPoewEcgA3XBOJCMSJaNI
pNe6fTux79lDHE1NxLlvH3GjmA9ClqamQJ9Kdft+ZaWNPXYs6JjFHItctrHxsXgmj3cRevUgKyQeDYLB p2XHDmLbu5fYa2vJxP79xIliHggN1tb6utTqew9LSqzs8eN+xyzmWOSyNTVPxVN4vMvQqwKpAfFw4A9u
zRMKM7+VyXqGa2sDjoYG4gIeo5F4IeBHMf/evWQKbqeoMO5Z3AfF0d4eudwn4fOpOHW+CiwGT45qKLgK plCY8oNc3nG/osJnr64mDuAymYgbAl4U8+7bR6bgdooK457FvV8c7e1QKDwyPp+KU+erwGLw7KgGgqsU
oXBVl1Taa9+yhbh27CAe4DMYiB8OJ1FwEhs4WVlJ/OvXE79KRbygJz/fnysQXML6XYA654FnxMPBXZ2Y Cle1pKd32rZuJY6dO4kLeIxG4oXDSRScxAZOlpQQ7/r1xKtWEzfoyMryZggEV7B+N6DOeeAF8WBwV8fG
mNUlkfT+WlAw4926lXjUauJhGOIWi4lbICDu2FjiXrSIWPj8QGdqqk+ZkEA3tAFkg+fBvOLhCLarXyIZ prbIZJ392dkz7m3biEujIS6GIU6xmDgFAuKMjCTORYvIIJ/va05I8KhiYuiGVoM08DKYVzwY/nbdlsl6
moCgA2JO4AJUOEhMDBmUSv/SZ2R8jfx3QA5YkHgw7HV1RrtKZXXweM+KAzp2Ny1txqTR3M9MTq7Ekniw xyFoh9gEcAAq7CcigvSkp/9lSE7+DvnvAylYkLg/bJWVJptabbHzeC+KAzrWl5g4Y9ZqH6bEx5dgSTRY
MHG/wXDQXVTkdsD9fOLj4CGwZmWRQa12+M01a3Kx9D9fK8HwGAxtrg0bPI64uGfEXeg9bRcVHwOW6Ghy mLjXaDzkzM112uF+PvEx8BhYUlNJj07X/86aNRlY+p+vFX+4jMZGx8aNLntU1AviDvSetouKj4Lh8HDS
B/Tn5gaGqqru6pVK2qb5i/h27TrqLCvzRnLuEonIo7Kyv83p6QEqPhISHwA94CbDzA7RfzJfERzFNkdR B37OyPD1lpY+MKhUtE3zF/Hs3n1sorDQHcq5QyQiTwoL/36QlOSj4kMB8W7QAdoZZraX/pP5iuAoNtpz
kc/B5weFn2rLkiXEXF7++6l16/oGNm2asGZmkl8geosKg++BiV5Xr579Ua+/XSSR0Bfkk1e9t76+lYo7 cz12Pt8v/FxbliwhA0VFv59eu7are/PmcUtKCrkP0TvgFvgRmEHb6tWzvxgM93JlMvqCfPaqd1dVNVDx
I4knJRFzQcEfjQzThdSmsuzst4YrKu6b0fuweDfoBB3ApFDM9lVVDXys0dCN54IojnfzZpszISGyc4g3 iVDicXFkYN26P2oYpgWptYVpae/2Fxc/HEDvg+KtoBk0AbNSOdtVWtr9mVZLN54LwjjuLVusEzExoZ1D
yGSdSDQCupHCfYWFrw3qdKO3c3Iei38DvgJXwVBu7swtne4ecpcDLseh1Zq9KSkRnRsZ5jskNYbEw18q vFoub0aiCdCNFO7PyXmrR68fuSeVPhX/HnwLroNemWzmjl4/iNzlgMux63QDbokkpHMTw9xEUk1APPil
7v7iYuVgVdWoKScn8G/xL6KiyC2pdMakVo8jrwQkcvpqa7dNqNUeD0TDzu+Ulv65J7J4OLjNhYX52FhL 4h7Iy1P1lJaO3JRKff8W/zosjNyRSmfMGs0Y8vJBLKeromL7uEbjckE06LyvoODPvaHFg8Gty8nJwsYO
v1RKroTETfn5s5+p1Y/KV678CDlzBRCLO6urj9kqKnyjSmWAOm+Wy7sxPp94OLiGtWtltB3dWGdSqWYv 305PJ9cC4uasrNkvNZonRStXfoqcuQKIxc1lZcetxcWeIZXKR53XKRStGJ9PPBhcY3a2nLYD3wyfWa2e
w/kLPF475rRgrkWIKCY1NX5Ap/v0QUWF61JxMU5fdDPGGcCn8zRpnuBeral5+2etdvymRuMuS0+n3wId vQrnr/B4FzGnA3MtQoQxCQnR3Xr9F4+Kix1X8vJw+sLrMM4APp2nSfME93p5+Xu/6nRjt7RaZ2FSEv0W
EAN6XB+vpT8SgBxsDF0X+vg/B+jRpCdnE3gFRHzgFgHqmH6s6ZXeLySoCfr+fzEELRgyxuH8Aw1h2aqt 6IEY0OP6dC39EQMUYFPgutDH/yVAjyY9OZvBGyDkA7cIUMf0Y02v9H4hQU3Q9/+rAWjBgDEO5x9IKtl+
epieAAAAAElFTkSuQmCC 4dDtOAAAAABJRU5ErkJggg==
</value> </value>
</data> </data>
<data name="BTT_DELETE.Image" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64"> <data name="BTT_DELETE.Image" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
<value> <value>
iVBORw0KGgoAAAANSUhEUgAAABgAAAAYCAYAAADgdz34AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8 iVBORw0KGgoAAAANSUhEUgAAABgAAAAYCAYAAADgdz34AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8
YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAAVBSURBVEhLjZVtTFNXGMcLQmdHO6AdarLSOcTxXqC39sVN YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAAVDSURBVEhLjZVtTFNXGMcLQmdHO6AdarLSOcTxXqC3lFs3
xfEiRQu2VNAxomZUXEHxBSWK0RhNNIrJkn3YPmwftsxkxi3RZVEmjJoxgUyROnFBQIe1lL4XLntJFunZ FceLFC3YgqBjRM2ouILiC0oUozGauPiSLNmH7cP2YctMZtwSXdzGhFE3J5ABAg4XBHRYS+l74bKXZJGe
/5RWZywbT/LL7T3nOf/n3+ecey+HhqW4OM6ckvLp9SVLRq5JJJ+sF4tfw3BMcPJ/YlihUP0mkXSPLl48 /U9pdcay8SS/3N5znvN//n3OuffyaAwUFkaZEhI+ur5s2eg1mezDDVLpSxiO8E/+T4yoVOxvMtmNsaVL
cCk11YChOBAVnKQxaDLxbxYXfz3c0kLGzp0jfc3NgWsqVZdSIFiO6djZrMgxxDBv27dsecSeOUOm29rI +y8nJlZgKAqE+SdpDBmNwp7Cwi9GmpvJ+PnzpLOpyXeNZTtyRaKVmI6cywodwwzzunXr1gfcmTNk5tw5
SFWV96P09AZJfHwipmeLdC1b9vH9ffvIw+PHydixY8SGZOvBg6RdpeqWCgRvIiVikVG1+h2IW9mzZwmL MlpZ6X4/ObleFh0di+m5Ih0rVnxwd/9+cv/ECTJ+/DixINl86BBpZdkbcpHoVaSELDKmVr8BcTN39izh
tWxrK5k+epTc12p934rFHyJFCGI410UiGxV/2NxMHh0+TGxImjh1itxuaQlcUyh+yhEI0pD4XBG0pcRe sJZraSEzx46Ru1qt5yup9D2kiEEE77pEYqHi95uayIMjR4gFSZOnT5Pe5mbfNZXqpwyRKAmJTxVBW4qs
XW2jzqn4NMTZAwfINIz5a2uJOSnpHtJUQMDpTE4+P1JXR8ZQwApshw4ROxY4T54kFrSrU6Xqzebz05Ec VVUW6pyKz0CcO3iQzMCYt6aGmOLi7iCNBSJee3z8hdHaWjKOAmZgOXyYWLHAfuoUuYV2tbNsV7pQmIxk
LDIsk5VMVFXZpk6fJizMTMMUi3XTe/eSyZ07SbtC4d4qEl1EahkQcvRLl77eLpd3927fHniMpHEwATdO f5ERhaJosrLSMv3uu4SDmRmY4rBuZt8+MrVrF2lVqZzbJJJLSC0BYp5++fKXW5XKGzd37PA9RNIEmIQb
7IkHLXPt30+uo0g+n59xm2GKxw0G+9SJE8GWsHDMor1sUxOZgrg5P9+bxuN9AWEjkAK64ZxYRiRKR5Fe O/bEhZY5Dhwg11EkWyhMucUwhRMVFdbpkyf9LeHgmEN7ucZGMg1xU3a2O0kg+BTCBiAHdMN5kYxEkowi
69atxL5rF3E0NhLnnj3EjWI+CN1tbAz0qdV3HlRU2NgjR4KOWcyxyGUbGp6KZ/B456FXB7JC4tEgGNx8 XeZt24h1925ia2gg9r17iRPFPBAabGjwdavVg/fKyizc0aN+xxzmOORy9fWPxVMEggvQqwVpAfFw4A9+
oTDje5msp7+mJuCorycu4DGZiBcCfhTz795NpuB2igrjnsV9UBzt7ZHLfVI+n4pT55lgIXh2VEPBVQiF tlic8p1C0dlTXe2z1dURB3AZjcQNAS+KeffsIdNwO02Fcc/h3i+O9nYqlR65UEjFqfNUsBg8OaqB4KvE
mZ25ub32TZuIa9s24gE+o5H44XASBSexgZMVFcS/di3xq9XEC3qUSn+eQHAB63cA6pwHXhAPB3dFYmJW 4tT2zMwu6+bNxLF9O3EBj8FAvHA4hYJT2MCpsjLiXbeOeNVq4gadubneLJHoItbvBNS5ADwjHgx+Tmxs
p1TaO6hUPvFu3kw8Gg3xMAxxi8XELRAQd2wscS9YQO7y+YGOlBSfKiGBbmg9yAYvgznFwxFs102p1DIB Wrtc3jWUk/PIvWULcWk0xMUwxCmVEqdIRJyRkcS5aBEZFAp9bQkJHjYmhm5oHUgHz4N5xYPhb1ePXD4w
QQfEnMAFqHCQmBgykJv7lyE9/QryPwA5YF7iwbDX1prsarXVweO9KA7o2GBq6hOzVvsgIzm5AkviwfzE CUEbxOzAAaiwn4gI0p+Z+VdFcvI3yH8HZIAFifvDWlNjtKrVZptA8Kw4oGNDiYmPTFrtvZT4+DIsiQYL
/UbjfndhodsB93OJj4NHwJqVRQZ0unvvrlyZh6X/+VoJhsdobHWtW+dxxMW9IO5C72m7qPgYGImOJoOg E/caDAec+flOG9zPJz4BHgBzWhrp1+nuvLlqVRaW/udrxR8ug6HFsX69yxYV9Yy4A72n7aLi42A0PJwM
Oy8vYKms/NWgUtE2zV3Et2PHYWdpqTeSc5dIRB6Xlv5tSUsLUPHhkHg/6AFmhpmx0H8yVxEcxVZHYaHP gR+ysnwD5eW/VrAsbdP8RTw7dx6xFxe7Qzl3SCTkYXHx37eSknxUfCQg3gc6gYlhZgfoP5mvCI5iiy0/
wecHhZ9ry6JFZKis7PcTq1f39W/YMGHNyCB3IXoL3AA/0gKUFStmfjYY7hRKpfQF+exV762ra6Hizkji 32MTCv3CT7VlyRIyXFLy+8k1a7r7Nm6cNKekkF8g2gtugh9pAUpOzuzPFRWD+XI5fUE+edW7a2ubqbg9
SUlkaM2aPxoYphOpjaXZ2e/dKy9/MITeh8W7QAdop0UUipm+ysr+z7VauvFcEMXxbtxocyYkRHYO8XqZ lHhcHBleu/aPeoZpR2pDcXr6W3dKS+8No/dB8Q7QBlppEZVqtru8vO8TrZZuPB+E8dybNlnsMTGhnUO8
rAOJJkA3UrinoOCtAb1+9E5OzlPxq+A7cBlYMjOf3NLr7yN3KeByHDrdkFciiejcxDA/IKkhJB7+UnH3 TqFoQ6IR0I0U783Le61frx8bzMh4LP4tuAqugIHU1Ee9ev1d5C4HfJ5Npxt2y2QhnRsZ5nsk1QfEg18q
FhWpBiorR6/m5AT+Lf5NVBS5hQJmjWYcecUgkdNXU7NlQqPxeCAadj5YUvLnrsji4eA2FRQosbEjN3Nz /r6CAra/vHzs64wM37/FvwwLI70oYNJoJpBXCGJ53dXVWyc1GpcLokHnQ0VFf+4OLR4MfmNeXi42drQn
yaWQuFmpnPlKo3lctnz5Z8iZLYBY2FFVdcRWXu4bVKkC1HmTXN6F8bnEw8E1rlolo+24gnVmtXrmIpy/ M5NcDoibcnNnP9doHpasXPkxcuYKIBa3VVYetZSWem6zrI86b1QqOzA+n3gw+IbVqxW0HVexzqRWz16C
wuO1YU4HZluEiGJSUuL79fovH5aXuy4UFeH0RTdhnAF8Ok+T5gju5erq93/R6cZvaLXu0rQ0+i3QAzGg 8xcEgnOY04G5FiHCmISE6D69/rP7paWOiwUFOH3hjRhngJDO06R5gn+lqurt2zrdxE2t1lmclES/BXog
x/XpWvojAcjB+tB1vo//S4AeTXpyNoA3QMQHbgGgjunHml7p/XyCmqDv/1dD0IIhYxzOP1xr2RYPr44i BfS4Pl5Lf8QAJdgQuC708X8O0KNJT85G8AoI+cAtAtQx/VjTK71fSFAT9P3/YgBaMGCMx/sHyjLY+hqD
AAAAAElFTkSuQmCC P/QAAAAASUVORK5CYII=
</value> </value>
</data> </data>
<data name="BTT_CLEAR_DONE.Image" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64"> <data name="BTT_CLEAR_DONE.Image" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
<value> <value>
iVBORw0KGgoAAAANSUhEUgAAABgAAAAYCAYAAADgdz34AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8 iVBORw0KGgoAAAANSUhEUgAAABgAAAAYCAYAAADgdz34AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8
YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAAVBSURBVEhLjZVtTFNXGMcLQmdHO6AdarLSOcTxXqC39sVN YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAAVDSURBVEhLjZVtTFNXGMcLQmdHO6AdarLSOcTxXqC3lFs3
xfEiRQu2VNAxomZUXEHxBSWK0RhNNIrJkn3YPmwftsxkxi3RZVEmjJoxgUyROnFBQIe1lL4XLntJFunZ FceLFC3YgqBjRM2ouILiC0oUozGauPiSLNmH7cP2YctMZtwSXdzGhFE3J5ABAg4XBHRYS+l74bKXZJGe
/5RWZywbT/LL7T3nOf/n3+ecey+HhqW4OM6ckvLp9SVLRq5JJJ+sF4tfw3BMcPJ/YlihUP0mkXSPLl48 /U9pdcay8SS/3N5znvN//n3OuffyaAwUFkaZEhI+ur5s2eg1mezDDVLpSxiO8E/+T4yoVOxvMtmNsaVL
cCk11YChOBAVnKQxaDLxbxYXfz3c0kLGzp0jfc3NgWsqVZdSIFiO6djZrMgxxDBv27dsecSeOUOm29rI +y8nJlZgKAqE+SdpDBmNwp7Cwi9GmpvJ+PnzpLOpyXeNZTtyRaKVmI6cywodwwzzunXr1gfcmTNk5tw5
SFWV96P09AZJfHwipmeLdC1b9vH9ffvIw+PHydixY8SGZOvBg6RdpeqWCgRvIiVikVG1+h2IW9mzZwmL MlpZ6X4/ObleFh0di+m5Ih0rVnxwd/9+cv/ECTJ+/DixINl86BBpZdkbcpHoVaSELDKmVr8BcTN39izh
tWxrK5k+epTc12p934rFHyJFCGI410UiGxV/2NxMHh0+TGxImjh1itxuaQlcUyh+yhEI0pD4XBG0pcRe sJZraSEzx46Ru1qt5yup9D2kiEEE77pEYqHi95uayIMjR4gFSZOnT5Pe5mbfNZXqpwyRKAmJTxVBW4qs
XW2jzqn4NMTZAwfINIz5a2uJOSnpHtJUQMDpTE4+P1JXR8ZQwApshw4ROxY4T54kFrSrU6Xqzebz05Ec VVUW6pyKz0CcO3iQzMCYt6aGmOLi7iCNBSJee3z8hdHaWjKOAmZgOXyYWLHAfuoUuYV2tbNsV7pQmIxk
LDIsk5VMVFXZpk6fJizMTMMUi3XTe/eSyZ07SbtC4d4qEl1EahkQcvRLl77eLpd3927fHniMpHEwATdO f5ERhaJosrLSMv3uu4SDmRmY4rBuZt8+MrVrF2lVqZzbJJJLSC0BYp5++fKXW5XKGzd37PA9RNIEmIQb
7IkHLXPt30+uo0g+n59xm2GKxw0G+9SJE8GWsHDMor1sUxOZgrg5P9+bxuN9AWEjkAK64ZxYRiRKR5Fe O/bEhZY5Dhwg11EkWyhMucUwhRMVFdbpkyf9LeHgmEN7ucZGMg1xU3a2O0kg+BTCBiAHdMN5kYxEkowi
69atxL5rF3E0NhLnnj3EjWI+CN1tbAz0qdV3HlRU2NgjR4KOWcyxyGUbGp6KZ/B456FXB7JC4tEgGNx8 XeZt24h1925ia2gg9r17iRPFPBAabGjwdavVg/fKyizc0aN+xxzmOORy9fWPxVMEggvQqwVpAfFw4A9+
oTDje5msp7+mJuCorycu4DGZiBcCfhTz795NpuB2igrjnsV9UBzt7ZHLfVI+n4pT55lgIXh2VEPBVQiF tlic8p1C0dlTXe2z1dURB3AZjcQNAS+KeffsIdNwO02Fcc/h3i+O9nYqlR65UEjFqfNUsBg8OaqB4KvE
mZ25ub32TZuIa9s24gE+o5H44XASBSexgZMVFcS/di3xq9XEC3qUSn+eQHAB63cA6pwHXhAPB3dFYmJW 4tT2zMwu6+bNxLF9O3EBj8FAvHA4hYJT2MCpsjLiXbeOeNVq4gadubneLJHoItbvBNS5ADwjHgx+Tmxs
p1TaO6hUPvFu3kw8Gg3xMAxxi8XELRAQd2wscS9YQO7y+YGOlBSfKiGBbmg9yAYvgznFwxFs102p1DIB Wrtc3jWUk/PIvWULcWk0xMUwxCmVEqdIRJyRkcS5aBEZFAp9bQkJHjYmhm5oHUgHz4N5xYPhb1ePXD4w
QQfEnMAFqHCQmBgykJv7lyE9/QryPwA5YF7iwbDX1prsarXVweO9KA7o2GBq6hOzVvsgIzm5AkviwfzE CUEbxOzAAaiwn4gI0p+Z+VdFcvI3yH8HZIAFifvDWlNjtKrVZptA8Kw4oGNDiYmPTFrtvZT4+DIsiQYL
/UbjfndhodsB93OJj4NHwJqVRQZ0unvvrlyZh6X/+VoJhsdobHWtW+dxxMW9IO5C72m7qPgYGImOJoOg E/caDAec+flOG9zPJz4BHgBzWhrp1+nuvLlqVRaW/udrxR8ug6HFsX69yxYV9Yy4A72n7aLi42A0PJwM
Oy8vYKms/NWgUtE2zV3Et2PHYWdpqTeSc5dIRB6Xlv5tSUsLUPHhkHg/6AFmhpmx0H8yVxEcxVZHYaHP gR+ysnwD5eW/VrAsbdP8RTw7dx6xFxe7Qzl3SCTkYXHx37eSknxUfCQg3gc6gYlhZgfoP5mvCI5iiy0/
wecHhZ9ry6JFZKis7PcTq1f39W/YMGHNyCB3IXoL3AA/0gKUFStmfjYY7hRKpfQF+exV762ra6Hizkji 32MTCv3CT7VlyRIyXFLy+8k1a7r7Nm6cNKekkF8g2gtugh9pAUpOzuzPFRWD+XI5fUE+edW7a2ubqbg9
SUlkaM2aPxoYphOpjaXZ2e/dKy9/MITeh8W7QAdop0UUipm+ysr+z7VauvFcEMXxbtxocyYkRHYO8XqZ lHhcHBleu/aPeoZpR2pDcXr6W3dKS+8No/dB8Q7QBlppEZVqtru8vO8TrZZuPB+E8dybNlnsMTGhnUO8
rAOJJkA3UrinoOCtAb1+9E5OzlPxq+A7cBlYMjOf3NLr7yN3KeByHDrdkFciiejcxDA/IKkhJB7+UnH3 TqFoQ6IR0I0U783Le61frx8bzMh4LP4tuAqugIHU1Ee9ev1d5C4HfJ5Npxt2y2QhnRsZ5nsk1QfEg18q
FhWpBiorR6/m5AT+Lf5NVBS5hQJmjWYcecUgkdNXU7NlQqPxeCAadj5YUvLnrsji4eA2FRQosbEjN3Nz /r6CAra/vHzs64wM37/FvwwLI70oYNJoJpBXCGJ53dXVWyc1GpcLokHnQ0VFf+4OLR4MfmNeXi42drQn
yaWQuFmpnPlKo3lctnz5Z8iZLYBY2FFVdcRWXu4bVKkC1HmTXN6F8bnEw8E1rlolo+24gnVmtXrmIpy/ M5NcDoibcnNnP9doHpasXPkxcuYKIBa3VVYetZSWem6zrI86b1QqOzA+n3gw+IbVqxW0HVexzqRWz16C
wuO1YU4HZluEiGJSUuL79fovH5aXuy4UFeH0RTdhnAF8Ok+T5gju5erq93/R6cZvaLXu0rQ0+i3QAzGg 8xcEgnOY04G5FiHCmISE6D69/rP7paWOiwUFOH3hjRhngJDO06R5gn+lqurt2zrdxE2t1lmclES/BXog
x/XpWvojAcjB+tB1vo//S4AeTXpyNoA3QMQHbgGgjunHml7p/XyCmqDv/1dD0IIhYxzOP1xr2RYPr44i BfS4Pl5Lf8QAJdgQuC708X8O0KNJT85G8AoI+cAtAtQx/VjTK71fSFAT9P3/YgBaMGCMx/sHyjLY+hqD
AAAAAElFTkSuQmCC P/QAAAAASUVORK5CYII=
</value> </value>
</data> </data>
<data name="BTT_CLEAR_ALL.Image" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64"> <data name="BTT_CLEAR_ALL.Image" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">
<value> <value>
iVBORw0KGgoAAAANSUhEUgAAABgAAAAYCAYAAADgdz34AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8 iVBORw0KGgoAAAANSUhEUgAAABgAAAAYCAYAAADgdz34AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8
YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAAVBSURBVEhLjZVtTFNXGMcLQmdHO6AdarLSOcTxXqC39sVN YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAAVDSURBVEhLjZVtTFNXGMcLQmdHO6AdarLSOcTxXqC3lFs3
xfEiRQu2VNAxomZUXEHxBSWK0RhNNIrJkn3YPmwftsxkxi3RZVEmjJoxgUyROnFBQIe1lL4XLntJFunZ FceLFC3YgqBjRM2ouILiC0oUozGauPiSLNmH7cP2YctMZtwSXdzGhFE3J5ABAg4XBHRYS+l74bKXZJGe
/5RWZywbT/LL7T3nOf/n3+ecey+HhqW4OM6ckvLp9SVLRq5JJJ+sF4tfw3BMcPJ/YlihUP0mkXSPLl48 /U9pdcay8SS/3N5znvN//n3OuffyaAwUFkaZEhI+ur5s2eg1mezDDVLpSxiO8E/+T4yoVOxvMtmNsaVL
cCk11YChOBAVnKQxaDLxbxYXfz3c0kLGzp0jfc3NgWsqVZdSIFiO6djZrMgxxDBv27dsecSeOUOm29rI +y8nJlZgKAqE+SdpDBmNwp7Cwi9GmpvJ+PnzpLOpyXeNZTtyRaKVmI6cywodwwzzunXr1gfcmTNk5tw5
SFWV96P09AZJfHwipmeLdC1b9vH9ffvIw+PHydixY8SGZOvBg6RdpeqWCgRvIiVikVG1+h2IW9mzZwmL MlpZ6X4/ObleFh0di+m5Ih0rVnxwd/9+cv/ECTJ+/DixINl86BBpZdkbcpHoVaSELDKmVr8BcTN39izh
tWxrK5k+epTc12p934rFHyJFCGI410UiGxV/2NxMHh0+TGxImjh1itxuaQlcUyh+yhEI0pD4XBG0pcRe sJZraSEzx46Ru1qt5yup9D2kiEEE77pEYqHi95uayIMjR4gFSZOnT5Pe5mbfNZXqpwyRKAmJTxVBW4qs
XW2jzqn4NMTZAwfINIz5a2uJOSnpHtJUQMDpTE4+P1JXR8ZQwApshw4ROxY4T54kFrSrU6Xqzebz05Ec VVUW6pyKz0CcO3iQzMCYt6aGmOLi7iCNBSJee3z8hdHaWjKOAmZgOXyYWLHAfuoUuYV2tbNsV7pQmIxk
LDIsk5VMVFXZpk6fJizMTMMUi3XTe/eSyZ07SbtC4d4qEl1EahkQcvRLl77eLpd3927fHniMpHEwATdO f5ERhaJosrLSMv3uu4SDmRmY4rBuZt8+MrVrF2lVqZzbJJJLSC0BYp5++fKXW5XKGzd37PA9RNIEmIQb
7IkHLXPt30+uo0g+n59xm2GKxw0G+9SJE8GWsHDMor1sUxOZgrg5P9+bxuN9AWEjkAK64ZxYRiRKR5Fe O/bEhZY5Dhwg11EkWyhMucUwhRMVFdbpkyf9LeHgmEN7ucZGMg1xU3a2O0kg+BTCBiAHdMN5kYxEkowi
69atxL5rF3E0NhLnnj3EjWI+CN1tbAz0qdV3HlRU2NgjR4KOWcyxyGUbGp6KZ/B456FXB7JC4tEgGNx8 XeZt24h1925ia2gg9r17iRPFPBAabGjwdavVg/fKyizc0aN+xxzmOORy9fWPxVMEggvQqwVpAfFw4A9+
oTDje5msp7+mJuCorycu4DGZiBcCfhTz795NpuB2igrjnsV9UBzt7ZHLfVI+n4pT55lgIXh2VEPBVQiF tlic8p1C0dlTXe2z1dURB3AZjcQNAS+KeffsIdNwO02Fcc/h3i+O9nYqlR65UEjFqfNUsBg8OaqB4KvE
mZ25ub32TZuIa9s24gE+o5H44XASBSexgZMVFcS/di3xq9XEC3qUSn+eQHAB63cA6pwHXhAPB3dFYmJW 4tT2zMwu6+bNxLF9O3EBj8FAvHA4hYJT2MCpsjLiXbeOeNVq4gadubneLJHoItbvBNS5ADwjHgx+Tmxs
p1TaO6hUPvFu3kw8Gg3xMAxxi8XELRAQd2wscS9YQO7y+YGOlBSfKiGBbmg9yAYvgznFwxFs102p1DIB Wrtc3jWUk/PIvWULcWk0xMUwxCmVEqdIRJyRkcS5aBEZFAp9bQkJHjYmhm5oHUgHz4N5xYPhb1ePXD4w
QQfEnMAFqHCQmBgykJv7lyE9/QryPwA5YF7iwbDX1prsarXVweO9KA7o2GBq6hOzVvsgIzm5AkviwfzE CUEbxOzAAaiwn4gI0p+Z+VdFcvI3yH8HZIAFifvDWlNjtKrVZptA8Kw4oGNDiYmPTFrtvZT4+DIsiQYL
/UbjfndhodsB93OJj4NHwJqVRQZ0unvvrlyZh6X/+VoJhsdobHWtW+dxxMW9IO5C72m7qPgYGImOJoOg E/caDAec+flOG9zPJz4BHgBzWhrp1+nuvLlqVRaW/udrxR8ug6HFsX69yxYV9Yy4A72n7aLi42A0PJwM
Oy8vYKms/NWgUtE2zV3Et2PHYWdpqTeSc5dIRB6Xlv5tSUsLUPHhkHg/6AFmhpmx0H8yVxEcxVZHYaHP gR+ysnwD5eW/VrAsbdP8RTw7dx6xFxe7Qzl3SCTkYXHx37eSknxUfCQg3gc6gYlhZgfoP5mvCI5iiy0/
wecHhZ9ry6JFZKis7PcTq1f39W/YMGHNyCB3IXoL3AA/0gKUFStmfjYY7hRKpfQF+exV762ra6Hizkji 32MTCv3CT7VlyRIyXFLy+8k1a7r7Nm6cNKekkF8g2gtugh9pAUpOzuzPFRWD+XI5fUE+edW7a2ubqbg9
SUlkaM2aPxoYphOpjaXZ2e/dKy9/MITeh8W7QAdop0UUipm+ysr+z7VauvFcEMXxbtxocyYkRHYO8XqZ lHhcHBleu/aPeoZpR2pDcXr6W3dKS+8No/dB8Q7QBlppEZVqtru8vO8TrZZuPB+E8dybNlnsMTGhnUO8
rAOJJkA3UrinoOCtAb1+9E5OzlPxq+A7cBlYMjOf3NLr7yN3KeByHDrdkFciiejcxDA/IKkhJB7+UnH3 TqFoQ6IR0I0U783Le61frx8bzMh4LP4tuAqugIHU1Ee9ev1d5C4HfJ5Npxt2y2QhnRsZ5nsk1QfEg18q
FhWpBiorR6/m5AT+Lf5NVBS5hQJmjWYcecUgkdNXU7NlQqPxeCAadj5YUvLnrsji4eA2FRQosbEjN3Nz /r6CAra/vHzs64wM37/FvwwLI70oYNJoJpBXCGJ53dXVWyc1GpcLokHnQ0VFf+4OLR4MfmNeXi42drQn
yaWQuFmpnPlKo3lctnz5Z8iZLYBY2FFVdcRWXu4bVKkC1HmTXN6F8bnEw8E1rlolo+24gnVmtXrmIpy/ M5NcDoibcnNnP9doHpasXPkxcuYKIBa3VVYetZSWem6zrI86b1QqOzA+n3gw+IbVqxW0HVexzqRWz16C
wuO1YU4HZluEiGJSUuL79fovH5aXuy4UFeH0RTdhnAF8Ok+T5gju5erq93/R6cZvaLXu0rQ0+i3QAzGg 8xcEgnOY04G5FiHCmISE6D69/rP7paWOiwUFOH3hjRhngJDO06R5gn+lqurt2zrdxE2t1lmclES/BXog
x/XpWvojAcjB+tB1vo//S4AeTXpyNoA3QMQHbgGgjunHml7p/XyCmqDv/1dD0IIhYxzOP1xr2RYPr44i BfS4Pl5Lf8QAJdgQuC708X8O0KNJT85G8AoI+cAtAtQx/VjTK71fSFAT9P3/YgBaMGCMx/sHyjLY+hqD
AAAAAElFTkSuQmCC P/QAAAAASUVORK5CYII=
</value> </value>
</data> </data>
<data name="BTT_LOG.Image" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64"> <data name="BTT_LOG.Image" type="System.Drawing.Bitmap, System.Drawing" mimetype="application/x-microsoft.net.object.bytearray.base64">

32
SCrawler.YouTube/Downloader/VideoListForm.vb
View File

@@ -81,7 +81,13 @@ Namespace DownloadObjects.STDownloader
If Not MyYouTubeSettings Is Nothing Then MyYouTubeSettings.Close() If Not MyYouTubeSettings Is Nothing Then MyYouTubeSettings.Close()
End Sub End Sub
Private Sub VideoListForm_KeyDown(sender As Object, e As KeyEventArgs) Handles Me.KeyDown Private Sub VideoListForm_KeyDown(sender As Object, e As KeyEventArgs) Handles Me.KeyDown
If e.KeyCode = Keys.Insert Then BTT_ADD.PerformClick() : e.Handled = True Dim b As Boolean = True
Select Case e.KeyCode
Case Keys.Insert : BTT_ADD.PerformClick()
Case Keys.F5 : BTT_DOWN.PerformClick()
Case Else : b = False
End Select
If b Then e.Handled = True
End Sub End Sub
#End Region #End Region
#Region "Refill, save list" #Region "Refill, save list"
@@ -230,10 +236,8 @@ Namespace DownloadObjects.STDownloader
BTT_ADD_NO_SHORTS.KeyClick, BTT_ADD_SHORTS_ONLY.KeyClick BTT_ADD_NO_SHORTS.KeyClick, BTT_ADD_SHORTS_ONLY.KeyClick
Dim pForm As ParsingProgressForm = Nothing Dim pForm As ParsingProgressForm = Nothing
Try Try
Dim canProcess As Boolean = True
If TP_CONTROLS.Controls.Count >= MyYouTubeSettings.ItemsListLimit Then canProcess = TP_CONTROLS.Controls.Cast(Of MediaItem).ListExists(ControlsDownloaded)
If canProcess Then
Dim useCookies As Boolean = MyYouTubeSettings.DefaultUseCookies Dim useCookies As Boolean = MyYouTubeSettings.DefaultUseCookies
Dim disableDown As Boolean = e.Shift
If e.Control Then useCookies = True If e.Control Then useCookies = True
Dim useCookiesParse As Boolean? = Nothing Dim useCookiesParse As Boolean? = Nothing
If useCookies Then useCookiesParse = True If useCookies Then useCookiesParse = True
@@ -296,18 +300,11 @@ Namespace DownloadObjects.STDownloader
If Not f Is Nothing Then If Not f Is Nothing Then
If TypeOf f Is IDesignXMLContainer Then DirectCast(f, IDesignXMLContainer).DesignXML = DesignXML If TypeOf f Is IDesignXMLContainer Then DirectCast(f, IDesignXMLContainer).DesignXML = DesignXML
f.ShowDialog() f.ShowDialog()
If f.DialogResult = DialogResult.OK Then If f.DialogResult = DialogResult.OK Then ControlCreateAndAdd(c, disableDown)
If TP_CONTROLS.Controls.Count >= MyYouTubeSettings.ItemsListLimit Then _
RemoveControls(TP_CONTROLS.Controls.Cast(Of MediaItem).LastOrDefault(ControlsDownloaded))
ControlCreateAndAdd(c)
End If
f.Dispose() f.Dispose()
End If End If
End If End If
End If End If
Else
MsgBoxE({$"Number of items to download exceeded!{vbCr}Reduce the number of items or increase the limit.", "New download"}, vbCritical)
End If
Catch oex As OperationCanceledException Catch oex As OperationCanceledException
Catch dex As ObjectDisposedException Catch dex As ObjectDisposedException
Catch ex As Exception Catch ex As Exception
@@ -425,8 +422,10 @@ Namespace DownloadObjects.STDownloader
UpdateLogButton() UpdateLogButton()
End Sub End Sub
Protected Sub AddToDownload(ByRef Item As MediaItem, ByVal RunThread As Boolean) Protected Sub AddToDownload(ByRef Item As MediaItem, ByVal RunThread As Boolean)
If MyJob.Count = 0 OrElse Not MyJob.Items.Exists(Function(i) i.MyContainer.GetHashCode) Then Dim hc% = Item.MyContainer.GetHashCode
Item.Pending = True If MyJob.Count = 0 OrElse Not MyJob.Items.Exists(Function(i) i.MyContainer.GetHashCode = hc) Then
'TODELETE: YT video downloader 'Item.Pending'
'Item.Pending = True
MyJob.Add(Item) MyJob.Add(Item)
Item.AddToQueue() Item.AddToQueue()
If RunThread Then StartDownloading() If RunThread Then StartDownloading()
@@ -475,7 +474,10 @@ Namespace DownloadObjects.STDownloader
Task.WaitAll(t.ToArray) Task.WaitAll(t.ToArray)
MyProgress.Perform(t.Count) MyProgress.Perform(t.Count)
If Indexes.Count > 0 Then If Indexes.Count > 0 Then
For i = Indexes.Count - 1 To 0 Step -1 : MyJob.Items.RemoveAt(Indexes(i)) : Next For i = Indexes.Count - 1 To 0 Step -1
MyJob.Item(Indexes(i)).Pending = False
MyJob.Items.RemoveAt(Indexes(i))
Next
End If End If
t.Clear() t.Clear()
End If End If

4
SCrawler.YouTube/My Project/AssemblyInfo.vb
View File

@@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices
' by using the '*' as shown below: ' by using the '*' as shown below:
' <Assembly: AssemblyVersion("1.0.*")> ' <Assembly: AssemblyVersion("1.0.*")>
<Assembly: AssemblyVersion("2023.5.12.0")> <Assembly: AssemblyVersion("2023.6.5.0")>
<Assembly: AssemblyFileVersion("2023.5.12.0")> <Assembly: AssemblyFileVersion("2023.6.5.0")>
<Assembly: NeutralResourcesLanguage("en")> <Assembly: NeutralResourcesLanguage("en")>

4
SCrawler.YouTube/Objects/YouTubeMediaContainerBase.vb
View File

@@ -440,7 +440,7 @@ Namespace API.YouTube.Objects
End Get End Get
End Property End Property
Protected _Exists As Boolean = True Protected _Exists As Boolean = True
Public ReadOnly Property Exists As Boolean Implements IDownloadableMedia.Exists Public Overridable ReadOnly Property Exists As Boolean Implements IDownloadableMedia.Exists
Get Get
If Not _Exists Then If Not _Exists Then
Return False Return False
@@ -643,6 +643,7 @@ Namespace API.YouTube.Objects
End If End If
If Not cmd.IsEmptyString Then If Not cmd.IsEmptyString Then
cmd = $"yt-dlp -f ""{cmd}""" cmd = $"yt-dlp -f ""{cmd}"""
If Not MyYouTubeSettings.ReplaceModificationDate Then cmd &= " --no-mtime"
cmd.StringAppend(formats, " ") cmd.StringAppend(formats, " ")
cmd.StringAppend(subs, " ") cmd.StringAppend(subs, " ")
cmd.StringAppend(YouTubeFunctions.GetCookiesCommand(WithCookies, YouTubeCookieNetscapeFile), " ") cmd.StringAppend(YouTubeFunctions.GetCookiesCommand(WithCookies, YouTubeCookieNetscapeFile), " ")
@@ -864,6 +865,7 @@ Namespace API.YouTube.Objects
.Information = $"Download {MediaType}" .Information = $"Download {MediaType}"
End With End With
End If End If
.MainProcessName = "yt-dlp"
.FileExchanger = MyCache.NewInstance(Of BatchFileExchanger)(CachePath, EDP.ReturnValue) .FileExchanger = MyCache.NewInstance(Of BatchFileExchanger)(CachePath, EDP.ReturnValue)
.FileExchanger.DeleteCacheOnDispose = True .FileExchanger.DeleteCacheOnDispose = True
.AddCommand("chcp 65001") .AddCommand("chcp 65001")

4
SCrawler.YouTubeDownloader/My Project/AssemblyInfo.vb
View File

@@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices
' by using the '*' as shown below: ' by using the '*' as shown below:
' <Assembly: AssemblyVersion("1.0.*")> ' <Assembly: AssemblyVersion("1.0.*")>
<Assembly: AssemblyVersion("2023.5.12.0")> <Assembly: AssemblyVersion("2023.6.5.0")>
<Assembly: AssemblyFileVersion("2023.5.12.0")> <Assembly: AssemblyFileVersion("2023.6.5.0")>
<Assembly: NeutralResourcesLanguage("en")> <Assembly: NeutralResourcesLanguage("en")>

11
SCrawler/API/Base/GDLBatch.vb
View File

@@ -72,15 +72,22 @@ Namespace API.Base.GDL
Friend Const UrlTextStart As String = UrlLibStart & " https" Friend Const UrlTextStart As String = UrlLibStart & " https"
Friend Sub New() Friend Sub New()
MyBase.New(True) MyBase.New(True)
MainProcessName = "gallery-dl"
ChangeDirectory(Settings.GalleryDLFile.File) ChangeDirectory(Settings.GalleryDLFile.File)
End Sub End Sub
Public Overrides Sub Create()
If TempPostsList Is Nothing Then TempPostsList = New List(Of String)
MyBase.Create()
End Sub
Protected Overrides Async Sub OutputDataReceiver(ByVal Sender As Object, ByVal e As DataReceivedEventArgs) Protected Overrides Async Sub OutputDataReceiver(ByVal Sender As Object, ByVal e As DataReceivedEventArgs)
If Not ProcessKilled Then
MyBase.OutputDataReceiver(Sender, e) MyBase.OutputDataReceiver(Sender, e)
Await Validate(e.Data) Await Validate(e.Data)
End If
End Sub End Sub
Protected Overridable Async Function Validate(ByVal Value As String) As Task Protected Overridable Async Function Validate(ByVal Value As String) As Task
If Await Task.Run(Of Boolean)(Function() Not Value.IsEmptyString AndAlso If Not ProcessKilled AndAlso Await Task.Run(Of Boolean)(Function() Not Value.IsEmptyString AndAlso
TempPostsList.Exists(Function(v) Value.Contains(v))) Then Kill(EDP.None) TempPostsList.Exists(Function(v) Value.Contains(v))) Then Kill()
End Function End Function
End Class End Class
End Namespace End Namespace

38
SCrawler/API/Base/UserDataBase.vb
View File

@@ -126,6 +126,7 @@ Namespace API.Base
Private Const Name_ReadyForDownload As String = "ReadyForDownload" Private Const Name_ReadyForDownload As String = "ReadyForDownload"
Private Const Name_DownloadImages As String = "DownloadImages" Private Const Name_DownloadImages As String = "DownloadImages"
Private Const Name_DownloadVideos As String = "DownloadVideos" Private Const Name_DownloadVideos As String = "DownloadVideos"
Private Const Name_IconBannerDownloaded As String = "IconBannerDownloaded"
Private Const Name_VideoCount As String = "VideoCount" Private Const Name_VideoCount As String = "VideoCount"
Private Const Name_PicturesCount As String = "PicturesCount" Private Const Name_PicturesCount As String = "PicturesCount"
@@ -434,6 +435,18 @@ BlockNullPicture:
Friend Property DownloadImages As Boolean = True Implements IUserData.DownloadImages Friend Property DownloadImages As Boolean = True Implements IUserData.DownloadImages
Friend Property DownloadVideos As Boolean = True Implements IUserData.DownloadVideos Friend Property DownloadVideos As Boolean = True Implements IUserData.DownloadVideos
Friend Property DownloadMissingOnly As Boolean = False Implements IUserData.DownloadMissingOnly Friend Property DownloadMissingOnly As Boolean = False Implements IUserData.DownloadMissingOnly
Private _IconBannerDownloaded As Boolean = False
Friend WriteOnly Property IconBannerDownloaded As Boolean
Set(ByVal IsDownloaded As Boolean)
If Not _IconBannerDownloaded = IsDownloaded Then _ForceSaveUserInfo = True
_IconBannerDownloaded = IsDownloaded
End Set
End Property
Friend ReadOnly Property DownloadIconBanner As Boolean
Get
Return Not _IconBannerDownloaded Or Settings.UpdateUserIconBannerEveryTime
End Get
End Property
#End Region #End Region
#Region "Content" #Region "Content"
Protected ReadOnly _ContentList As List(Of UserMedia) Protected ReadOnly _ContentList As List(Of UserMedia)
@@ -751,6 +764,7 @@ BlockNullPicture:
ReadyForDownload = x.Value(Name_ReadyForDownload).FromXML(Of Boolean)(True) ReadyForDownload = x.Value(Name_ReadyForDownload).FromXML(Of Boolean)(True)
DownloadImages = x.Value(Name_DownloadImages).FromXML(Of Boolean)(True) DownloadImages = x.Value(Name_DownloadImages).FromXML(Of Boolean)(True)
DownloadVideos = x.Value(Name_DownloadVideos).FromXML(Of Boolean)(True) DownloadVideos = x.Value(Name_DownloadVideos).FromXML(Of Boolean)(True)
_IconBannerDownloaded = x.Value(Name_IconBannerDownloaded).FromXML(Of Boolean)(False)
DownloadedVideos(True) = x.Value(Name_VideoCount).FromXML(Of Integer)(0) DownloadedVideos(True) = x.Value(Name_VideoCount).FromXML(Of Integer)(0)
DownloadedPictures(True) = x.Value(Name_PicturesCount).FromXML(Of Integer)(0) DownloadedPictures(True) = x.Value(Name_PicturesCount).FromXML(Of Integer)(0)
LastUpdated = AConvert(Of Date)(x.Value(Name_LastUpdated), ADateTime.Formats.BaseDateTime, Nothing) LastUpdated = AConvert(Of Date)(x.Value(Name_LastUpdated), ADateTime.Formats.BaseDateTime, Nothing)
@@ -799,6 +813,7 @@ BlockNullPicture:
x.Add(Name_ReadyForDownload, ReadyForDownload.BoolToInteger) x.Add(Name_ReadyForDownload, ReadyForDownload.BoolToInteger)
x.Add(Name_DownloadImages, DownloadImages.BoolToInteger) x.Add(Name_DownloadImages, DownloadImages.BoolToInteger)
x.Add(Name_DownloadVideos, DownloadVideos.BoolToInteger) x.Add(Name_DownloadVideos, DownloadVideos.BoolToInteger)
x.Add(Name_IconBannerDownloaded, _IconBannerDownloaded.BoolToInteger)
x.Add(Name_VideoCount, DownloadedVideos(True)) x.Add(Name_VideoCount, DownloadedVideos(True))
x.Add(Name_PicturesCount, DownloadedPictures(True)) x.Add(Name_PicturesCount, DownloadedPictures(True))
x.Add(Name_LastUpdated, AConvert(Of String)(LastUpdated, ADateTime.Formats.BaseDateTime, String.Empty)) x.Add(Name_LastUpdated, AConvert(Of String)(LastUpdated, ADateTime.Formats.BaseDateTime, String.Empty))
@@ -934,12 +949,12 @@ BlockNullPicture:
_EnvirUserExists = UserExists _EnvirUserExists = UserExists
_EnvirUserSuspended = UserSuspended _EnvirUserSuspended = UserSuspended
_EnvirChanged = False _EnvirChanged = False
_EnvirInvokeUserUpdated = False
UserExists = True UserExists = True
UserSuspended = False UserSuspended = False
DownloadedPictures(False) = 0 DownloadedPictures(False) = 0
DownloadedVideos(False) = 0 DownloadedVideos(False) = 0
_PictureExists = Settings.ViewModeIsPicture AndAlso Not GetPicture(Of Image)(False) Is Nothing _PictureExists = Settings.ViewModeIsPicture AndAlso Not GetPicture(Of Image)(False) Is Nothing
_EnvirInvokeUserUpdated = False
End Sub End Sub
Private Sub EnvirChanged(ByVal NewValue As Object, <CallerMemberName> Optional ByVal Caller As String = Nothing) Private Sub EnvirChanged(ByVal NewValue As Object, <CallerMemberName> Optional ByVal Caller As String = Nothing)
If _DownloadInProgress Then If _DownloadInProgress Then
@@ -997,8 +1012,10 @@ BlockNullPicture:
If UseMD5Comparison Then ValidateMD5(Token) : ProgressPre.Done() : ThrowAny(Token) If UseMD5Comparison Then ValidateMD5(Token) : ProgressPre.Done() : ThrowAny(Token)
If _TempPostsList.Count > 0 And Not DownloadMissingOnly And __SaveData Then _ If _TempPostsList.Count > 0 And Not DownloadMissingOnly And __SaveData Then
If _TempPostsList.Count > 1000 Then _TempPostsList.ListAddList(_TempPostsList.ListTake(-2, 1000, EDP.ReturnValue).ListReverse, LAP.ClearBeforeAdd)
TextSaver.SaveTextToFile(_TempPostsList.ListToString(Environment.NewLine), MyFilePosts, True,, EDP.None) TextSaver.SaveTextToFile(_TempPostsList.ListToString(Environment.NewLine), MyFilePosts, True,, EDP.None)
End If
_ContentNew.ListAddList(_TempMediaList, LAP.ClearBeforeAdd) _ContentNew.ListAddList(_TempMediaList, LAP.ClearBeforeAdd)
DownloadContent(Token) DownloadContent(Token)
ThrowIfDisposed() ThrowIfDisposed()
@@ -1079,7 +1096,8 @@ BlockNullPicture:
Progress = Data.Progress Progress = Data.Progress
If Not Responser Is Nothing Then Responser.Dispose() If Not Responser Is Nothing Then Responser.Dispose()
Responser = New Responser Responser = New Responser
If Not HOST Is Nothing AndAlso Not HOST.Responser Is Nothing Then Responser.Copy(HOST.Responser) If Not HOST Is Nothing AndAlso HOST.Available(ISiteSettings.Download.SingleObject, True) AndAlso
Not HOST.Responser Is Nothing Then Responser.Copy(HOST.Responser)
SeparateVideoFolder = False SeparateVideoFolder = False
IsSingleObjectDownload = True IsSingleObjectDownload = True
DownloadSingleObject_GetPosts(Data, Token) DownloadSingleObject_GetPosts(Data, Token)
@@ -1338,6 +1356,16 @@ BlockNullPicture:
Dim f As SFile Dim f As SFile
Dim v As UserMedia Dim v As UserMedia
Dim fileNumProvider As SFileNumbers = SFileNumbers.Default Dim fileNumProvider As SFileNumbers = SFileNumbers.Default
Dim __deleteFile As Action(Of SFile, String) = Sub(ByVal FileToDelete As SFile, ByVal FileUrl As String)
Try
If FileToDelete.Exists Then FileToDelete.Delete(,, EDP.ThrowException)
Catch file_io_ex As IOException
MyMainLOG = "File download aborted. You should download the following file again." & vbCr &
$"File: {FileToDelete}{vbCr}URL: {FileUrl}"
Catch file_del_ex As Exception
ErrorsDescriber.Execute(EDP.SendToLog, file_del_ex)
End Try
End Sub
Using w As New OptionalWebClient(Me) Using w As New OptionalWebClient(Me)
If vsf Then CSFileP($"{MyDir}\Video\").Exists(SFO.Path) If vsf Then CSFileP($"{MyDir}\Video\").Exists(SFO.Path)
@@ -1426,7 +1454,9 @@ BlockNullPicture:
DownloadContentDefault_PostProcessing(v, f, Token) DownloadContentDefault_PostProcessing(v, f, Token)
dCount += 1 dCount += 1
Catch woex As OperationCanceledException When Token.IsCancellationRequested Catch woex As OperationCanceledException When Token.IsCancellationRequested
If f.Exists Then f.Delete() 'TODELETE: UserDataBase.DownloadContentDefault: remove file when 'OperationCanceledException'
'If f.Exists Then f.Delete(,, EDP.SendToLog)
__deleteFile.Invoke(f, v.URL_BASE)
v.State = UStates.Missing v.State = UStates.Missing
v.Attempts += 1 v.Attempts += 1
_ContentNew(i) = v _ContentNew(i) = v

1
SCrawler/API/Instagram/Declarations.vb
View File

@@ -46,6 +46,7 @@ Namespace API.Instagram
If Not token.IsEmptyString Then Destination.Headers.Add(SiteSettings.Header_CSRF_TOKEN, token) If Not token.IsEmptyString Then Destination.Headers.Add(SiteSettings.Header_CSRF_TOKEN, token)
If Not isInternal Then If Not isInternal Then
Destination.Cookies.Update(Source.Cookies, CookieKeeper.UpdateModes.ReplaceByNameAll, False, EDP.SendToLog) Destination.Cookies.Update(Source.Cookies, CookieKeeper.UpdateModes.ReplaceByNameAll, False, EDP.SendToLog)
Destination.Cookies.Update(EDP.SendToLog)
Destination.SaveSettings() Destination.SaveSettings()
End If End If
End If End If

116
SCrawler/API/Instagram/SiteSettings.vb
View File

@@ -70,33 +70,54 @@ Namespace API.Instagram
End Class End Class
#End Region #End Region
#Region "Authorization properties" #Region "Authorization properties"
<PropertyOption(ControlText:="Hash", ControlToolTip:="Instagram session hash for tagged posts", IsAuth:=True), PXML("InstaHash"), ControlNumber(0)>
Friend ReadOnly Property HashTagged As PropertyValue
<PropertyOption(ControlText:="x-csrftoken", IsAuth:=True, AllowNull:=False), ControlNumber(2)>
Friend ReadOnly Property CSRF_TOKEN As PropertyValue
<PropertyOption(ControlText:="x-ig-app-id", IsAuth:=True, AllowNull:=False), ControlNumber(3)>
Friend Property IG_APP_ID As PropertyValue
<PropertyOption(ControlText:="x-ig-www-claim", IsAuth:=True, AllowNull:=True), ControlNumber(4)>
Friend Property IG_WWW_CLAIM As PropertyValue
Friend Overrides Function BaseAuthExists() As Boolean
Return Responser.CookiesExists And ACheck(IG_APP_ID.Value) And ACheck(CSRF_TOKEN.Value)
End Function
Private Const Header_IG_APP_ID As String = "x-ig-app-id" Private Const Header_IG_APP_ID As String = "x-ig-app-id"
Friend Const Header_IG_WWW_CLAIM As String = "x-ig-www-claim" Friend Const Header_IG_WWW_CLAIM As String = "x-ig-www-claim"
Friend Const Header_CSRF_TOKEN As String = "x-csrftoken" Friend Const Header_CSRF_TOKEN As String = "x-csrftoken"
Private Const Header_ASBD_ID As String = "X-Asbd-Id"
Private ReadOnly Header_Browser As New HttpHeader("Sec-Ch-Ua", """Google Chrome"";v=""113"", ""Chromium"";v=""113"", ""Not-A.Brand"";v=""24""")
Private ReadOnly Header_BrowserExt As New HttpHeader("Sec-Ch-Ua-Full-Version-List", """Google Chrome"";v=""113.0.5672.127"", ""Chromium"";v=""113.0.5672.127"", ""Not-A.Brand"";v=""24.0.0.0""")
Private ReadOnly Header_Platform As New HttpHeader("Sec-Ch-Ua-Platform-Version", """10.0.0""")
<PropertyOption(ControlText:="Hash", ControlToolTip:="Instagram session hash for tagged posts", IsAuth:=True), PXML("InstaHash"), ControlNumber(0)>
Friend ReadOnly Property HashTagged As PropertyValue
<PropertyOption(ControlText:="x-csrftoken", IsAuth:=True, AllowNull:=False), ControlNumber(2)>
Friend ReadOnly Property HH_CSRF_TOKEN As PropertyValue
<PropertyOption(ControlText:="x-ig-app-id", IsAuth:=True, AllowNull:=False), ControlNumber(3)>
Friend Property HH_IG_APP_ID As PropertyValue
<PropertyOption(ControlText:="x-asbd-id", IsAuth:=True, AllowNull:=True), ControlNumber(4)>
Friend Property HH_ASBD_ID As PropertyValue
<PropertyOption(ControlText:="x-ig-www-claim", IsAuth:=True, AllowNull:=True), ControlNumber(5)>
Friend Property HH_IG_WWW_CLAIM As PropertyValue
<PropertyOption(ControlText:="sec-ch-ua", IsAuth:=True, AllowNull:=True), ControlNumber(6)>
Private Property HH_BROWSER As PropertyValue
<PropertyOption(ControlText:="sec-ch-ua-full", ControlToolTip:="sec-ch-ua-full-version-list", IsAuth:=True, AllowNull:=True), ControlNumber(7)>
Private Property HH_BROWSER_EXT As PropertyValue
<PropertyOption(ControlText:="sec-ch-ua-platform-ver", ControlToolTip:="sec-ch-ua-platform-version", IsAuth:=True, AllowNull:=True), ControlNumber(8)>
Private Property HH_PLATFORM As PropertyValue
<PropertyOption(ControlText:="UserAgent", IsAuth:=True, AllowNull:=True), ControlNumber(9)>
Private Property HH_USER_AGENT As PropertyValue
Friend Overrides Function BaseAuthExists() As Boolean
Return Responser.CookiesExists And ACheck(HH_IG_APP_ID.Value) And ACheck(HH_CSRF_TOKEN.Value)
End Function
Private _FieldsChangerSuspended As Boolean = False Private _FieldsChangerSuspended As Boolean = False
Private Sub ChangeResponserFields(ByVal PropName As String, ByVal Value As Object) Private Sub ChangeResponserFields(ByVal PropName As String, ByVal Value As Object)
If Not _FieldsChangerSuspended And Not PropName.IsEmptyString Then If Not _FieldsChangerSuspended And Not PropName.IsEmptyString Then
Dim f$ = String.Empty Dim f$ = String.Empty
Dim isUserAgent As Boolean = False
Select Case PropName Select Case PropName
Case NameOf(IG_APP_ID) : f = Header_IG_APP_ID Case NameOf(HH_IG_APP_ID) : f = Header_IG_APP_ID
Case NameOf(IG_WWW_CLAIM) : f = Header_IG_WWW_CLAIM Case NameOf(HH_ASBD_ID) : f = Header_ASBD_ID
Case NameOf(CSRF_TOKEN) : f = Header_CSRF_TOKEN Case NameOf(HH_IG_WWW_CLAIM) : f = Header_IG_WWW_CLAIM
Case NameOf(HH_CSRF_TOKEN) : f = Header_CSRF_TOKEN
Case NameOf(HH_BROWSER) : f = Header_Browser.Name
Case NameOf(HH_BROWSER_EXT) : f = Header_BrowserExt.Name
Case NameOf(HH_PLATFORM) : f = Header_Platform.Name
Case NameOf(HH_USER_AGENT) : isUserAgent = True
End Select End Select
If Not f.IsEmptyString Then If Not f.IsEmptyString Then
Responser.Headers.Remove(f) Responser.Headers.Remove(f)
If Not CStr(Value).IsEmptyString Then Responser.Headers.Add(f, CStr(Value)) If Not CStr(Value).IsEmptyString Then Responser.Headers.Add(f, CStr(Value))
Responser.SaveSettings() ElseIf isUserAgent Then
Responser.UserAgent = CStr(Value)
End If End If
End If End If
End Sub End Sub
@@ -192,13 +213,53 @@ Namespace API.Instagram
Dim app_id$ = String.Empty Dim app_id$ = String.Empty
Dim www_claim$ = String.Empty Dim www_claim$ = String.Empty
Dim token$ = String.Empty Dim token$ = String.Empty
Dim asbd$ = String.Empty
Dim browser$ = String.Empty
Dim browserExt$ = String.Empty
Dim platform$ = String.Empty
Dim useragent$ = String.Empty
Dim __UpdateHeader As Action(Of HttpHeader, Boolean) = Sub(ByVal h As HttpHeader, ByVal UpdateValueIfEmpty As Boolean)
With Responser.Headers
Dim i% = .IndexOf(h)
Dim hh As HttpHeader
If i >= 0 Then
hh = .Item(i)
If hh.Value.IsEmptyString And UpdateValueIfEmpty Then hh.Value = h.Value
Else
hh = h
End If
.Add(hh)
End With
End Sub
With Responser With Responser
If .Headers.Count > 0 Then .Accept = "*/*"
token = .Headers.Value(Header_CSRF_TOKEN) useragent = .UserAgent
app_id = .Headers.Value(Header_IG_APP_ID) With .Headers
www_claim = .Headers.Value(Header_IG_WWW_CLAIM) If .Count > 0 Then
token = .Value(Header_CSRF_TOKEN)
app_id = .Value(Header_IG_APP_ID)
www_claim = .Value(Header_IG_WWW_CLAIM)
asbd = .Value(Header_ASBD_ID)
browser = .Value(Header_Browser.Name)
browserExt = .Value(Header_BrowserExt.Name)
platform = .Value(Header_Platform.Name)
End If End If
.Add("Dnt", 1)
__UpdateHeader(Header_Browser, browser.IsEmptyString)
browser = .Value(Header_Browser.Name)
__UpdateHeader(Header_BrowserExt, browserExt.IsEmptyString)
browserExt = .Value(Header_BrowserExt.Name)
.Add("Sec-Ch-Ua-Mobile", "?0")
.Add("Sec-Ch-Ua-Platform", """Windows""")
__UpdateHeader(Header_Platform, platform.IsEmptyString)
platform = .Value(Header_Platform.Name)
.Add("Sec-Fetch-Dest", "empty")
.Add("Sec-Fetch-Mode", "cors")
.Add("Sec-Fetch-Site", "same-origin")
.Add("X-Requested-With", "XMLHttpRequest")
End With
.CookiesExtractMode = Responser.CookiesExtractModes.Response .CookiesExtractMode = Responser.CookiesExtractModes.Response
.CookiesUpdateMode = CookieKeeper.UpdateModes.ReplaceByNameAll .CookiesUpdateMode = CookieKeeper.UpdateModes.ReplaceByNameAll
.CookiesExtractedAutoSave = False .CookiesExtractedAutoSave = False
@@ -207,9 +268,14 @@ Namespace API.Instagram
Dim n() As String = {SettingsCLS.Name_Node_Sites, Site.ToString} Dim n() As String = {SettingsCLS.Name_Node_Sites, Site.ToString}
HashTagged = New PropertyValue(String.Empty, GetType(String)) HashTagged = New PropertyValue(String.Empty, GetType(String))
CSRF_TOKEN = New PropertyValue(token, GetType(String), Sub(v) ChangeResponserFields(NameOf(CSRF_TOKEN), v)) HH_CSRF_TOKEN = New PropertyValue(token, GetType(String), Sub(v) ChangeResponserFields(NameOf(HH_CSRF_TOKEN), v))
IG_APP_ID = New PropertyValue(app_id, GetType(String), Sub(v) ChangeResponserFields(NameOf(IG_APP_ID), v)) HH_IG_APP_ID = New PropertyValue(app_id, GetType(String), Sub(v) ChangeResponserFields(NameOf(HH_IG_APP_ID), v))
IG_WWW_CLAIM = New PropertyValue(www_claim.IfNullOrEmpty(0), GetType(String), Sub(v) ChangeResponserFields(NameOf(IG_WWW_CLAIM), v)) HH_ASBD_ID = New PropertyValue(asbd, GetType(String), Sub(v) ChangeResponserFields(NameOf(HH_ASBD_ID), v))
HH_IG_WWW_CLAIM = New PropertyValue(www_claim.IfNullOrEmpty(0), GetType(String), Sub(v) ChangeResponserFields(NameOf(HH_IG_WWW_CLAIM), v))
HH_BROWSER = New PropertyValue(browser, GetType(String), Sub(v) ChangeResponserFields(NameOf(HH_BROWSER), v))
HH_BROWSER_EXT = New PropertyValue(browserExt, GetType(String), Sub(v) ChangeResponserFields(NameOf(HH_BROWSER_EXT), v))
HH_PLATFORM = New PropertyValue(platform, GetType(String), Sub(v) ChangeResponserFields(NameOf(HH_PLATFORM), v))
HH_USER_AGENT = New PropertyValue(useragent, GetType(String), Sub(v) ChangeResponserFields(NameOf(HH_USER_AGENT), v))
DownloadTimeline = New PropertyValue(True) DownloadTimeline = New PropertyValue(True)
DownloadStories = New PropertyValue(True) DownloadStories = New PropertyValue(True)
@@ -275,7 +341,7 @@ Namespace API.Instagram
Private _NextTagged As Boolean = True Private _NextTagged As Boolean = True
Friend Overrides Sub DownloadStarted(ByVal What As Download) Friend Overrides Sub DownloadStarted(ByVal What As Download)
ActiveJobs += 1 ActiveJobs += 1
If LastDownloadDate.Value.AddMinutes(120) < Now Or Not ACheck(IG_WWW_CLAIM.Value) Then IG_WWW_CLAIM.Value = "0" If LastDownloadDate.Value.AddMinutes(120) < Now Or Not ACheck(HH_IG_WWW_CLAIM.Value) Then HH_IG_WWW_CLAIM.Value = "0"
End Sub End Sub
Friend Overrides Sub BeforeStartDownload(ByVal User As Object, ByVal What As Download) Friend Overrides Sub BeforeStartDownload(ByVal User As Object, ByVal What As Download)
With DirectCast(User, UserData) With DirectCast(User, UserData)
@@ -299,8 +365,8 @@ Namespace API.Instagram
LastDownloadDate.Value = Now LastDownloadDate.Value = Now
LastRequestsCount.Value = .RequestsCount LastRequestsCount.Value = .RequestsCount
_FieldsChangerSuspended = True _FieldsChangerSuspended = True
IG_WWW_CLAIM.Value = Responser.Headers.Value(Header_IG_WWW_CLAIM) HH_IG_WWW_CLAIM.Value = Responser.Headers.Value(Header_IG_WWW_CLAIM)
CSRF_TOKEN.Value = Responser.Headers.Value(Header_CSRF_TOKEN) HH_CSRF_TOKEN.Value = Responser.Headers.Value(Header_CSRF_TOKEN)
_FieldsChangerSuspended = False _FieldsChangerSuspended = False
End With End With
End Sub End Sub

44
SCrawler/API/Mastodon/UserData.vb
View File

@@ -13,6 +13,7 @@ Imports PersonalUtilities.Functions.XML
Imports PersonalUtilities.Functions.RegularExpressions Imports PersonalUtilities.Functions.RegularExpressions
Imports PersonalUtilities.Tools.Web.Documents.JSON Imports PersonalUtilities.Tools.Web.Documents.JSON
Imports UTypes = SCrawler.API.Base.UserMedia.Types Imports UTypes = SCrawler.API.Base.UserMedia.Types
Imports UStates = SCrawler.API.Base.UserMedia.States
Namespace API.Mastodon Namespace API.Mastodon
Friend Class UserData : Inherits Twitter.UserData Friend Class UserData : Inherits Twitter.UserData
#Region "XML names" #Region "XML names"
@@ -136,12 +137,15 @@ Namespace API.Mastodon
If __imgFile.Extension.IsEmptyString Then __imgFile.Extension = "jpg" If __imgFile.Extension.IsEmptyString Then __imgFile.Extension = "jpg"
__imgFile.Path = MyFile.CutPath.Path __imgFile.Path = MyFile.CutPath.Path
If Not __imgFile.Exists Then GetWebFile(img, __imgFile, EDP.None) If Not __imgFile.Exists Then GetWebFile(img, __imgFile, EDP.None)
If __imgFile.Exists Then IconBannerDownloaded = True
End If End If
End If End If
End Sub End Sub
If DownloadIconBanner Then
__getImage.Invoke(.Value("header").IfNullOrEmpty(.Value("header_static"))) __getImage.Invoke(.Value("header").IfNullOrEmpty(.Value("header_static")))
__getImage.Invoke(.Value("avatar").IfNullOrEmpty(.Value("avatar_static"))) __getImage.Invoke(.Value("avatar").IfNullOrEmpty(.Value("avatar_static")))
End If End If
End If
End With End With
End If End If
@@ -235,8 +239,44 @@ Namespace API.Mastodon
Return $"https://{Domain}/api/v1/statuses/" & "{0}" Return $"https://{Domain}/api/v1/statuses/" & "{0}"
End Function End Function
Protected Overrides Sub ReparseMissing(ByVal Token As CancellationToken) Protected Overrides Sub ReparseMissing(ByVal Token As CancellationToken)
SinglePostUrl = GetSinglePostPattern(MyCredentials.Domain) Dim rList As New List(Of Integer)
MyBase.ReparseMissing(Token) Dim URL$ = String.Empty
Try
If ContentMissingExists Then
Dim SinglePostUrl$ = GetSinglePostPattern(MyCredentials.Domain)
Dim m As UserMedia
Dim r$, PostDate$
Dim j As EContainer
ProgressPre.ChangeMax(_ContentList.Count)
For i% = 0 To _ContentList.Count - 1
ProgressPre.Perform()
If _ContentList(i).State = UStates.Missing Then
m = _ContentList(i)
If Not m.Post.ID.IsEmptyString Then
ThrowAny(Token)
URL = String.Format(SinglePostUrl, m.Post.ID)
r = Responser.GetResponse(URL,, EDP.ReturnValue)
If Not r.IsEmptyString Then
j = JsonDocument.Parse(r)
If Not j Is Nothing Then
PostDate = String.Empty
If j.Contains("created_at") Then PostDate = j("created_at").Value Else PostDate = String.Empty
ObtainMedia(j, m.Post.ID, PostDate, UStates.Missing)
rList.Add(i)
End If
End If
End If
End If
Next
End If
Catch ex As Exception
ProcessException(ex, Token, $"ReparseMissing error [{URL}]")
Finally
If rList.Count > 0 Then
For i% = rList.Count - 1 To 0 Step -1 : _ContentList.RemoveAt(i) : Next
rList.Clear()
End If
End Try
End Sub End Sub
#End Region #End Region
#Region "DownloadSingleObject" #Region "DownloadSingleObject"

4
SCrawler/API/Pinterest/UserData.vb
View File

@@ -270,11 +270,11 @@ Namespace API.Pinterest
If IsBoardsRequested Then If IsBoardsRequested Then
If ErrorOutputData.Count > 0 Then If ErrorOutputData.Count > 0 Then
If Await Task.Run(Of Boolean)(Function() ErrorOutputData.Exists(Function(ee) Not ee.IsEmptyString AndAlso If Await Task.Run(Of Boolean)(Function() ErrorOutputData.Exists(Function(ee) Not ee.IsEmptyString AndAlso
ee.StartsWith(UrlTextStart))) Then Kill(EDP.None) ee.StartsWith(UrlTextStart))) Then Kill()
End If End If
Else Else
If Await Task.Run(Of Boolean)(Function() Not Value.IsEmptyString AndAlso If Await Task.Run(Of Boolean)(Function() Not Value.IsEmptyString AndAlso
Source._TempPostsList.Exists(Function(v) Value.Contains(v))) Then Kill(EDP.None) Source._TempPostsList.Exists(Function(v) Value.Contains(v))) Then Kill()
End If End If
End Function End Function
End Class End Class

3
SCrawler/API/Reddit/UserData.vb
View File

@@ -427,11 +427,14 @@ Namespace API.Reddit
If f.Extension.IsEmptyString Then f.Extension = "jpg" If f.Extension.IsEmptyString Then f.Extension = "jpg"
f.Path = dir.Path f.Path = dir.Path
If Not f.Exists Then GetWebFile(img, f, EDP.ReturnValue) If Not f.Exists Then GetWebFile(img, f, EDP.ReturnValue)
If f.Exists Then IconBannerDownloaded = True
End If End If
End If End If
End Sub End Sub
If DownloadIconBanner Then
__getFile.Invoke(.Value("icon_img")) __getFile.Invoke(.Value("icon_img"))
__getFile.Invoke(.Value("banner_img")) __getFile.Invoke(.Value("banner_img"))
End If
End With End With
End If End If
End Using End Using

1
SCrawler/API/Twitter/Declarations.vb
View File

@@ -14,7 +14,6 @@ Namespace API.Twitter
Friend Const TwitterSite As String = "Twitter" Friend Const TwitterSite As String = "Twitter"
Friend Const TwitterSiteKey As String = "AndyProgram_Twitter" Friend Const TwitterSiteKey As String = "AndyProgram_Twitter"
Friend ReadOnly DateProvider As ADateTime = GetDateProvider() Friend ReadOnly DateProvider As ADateTime = GetDateProvider()
Friend ReadOnly VideoNode As NodeParams() = {New NodeParams("video_info", True, True, True, True, 10)}
Friend ReadOnly VideoSizeRegEx As RParams = RParams.DMS("\d+x(\d+)", 1, EDP.ReturnValue) Friend ReadOnly VideoSizeRegEx As RParams = RParams.DMS("\d+x(\d+)", 1, EDP.ReturnValue)
Private Function GetDateProvider() As ADateTime Private Function GetDateProvider() As ADateTime
Dim n As DateTimeFormatInfo = CultureInfo.GetCultureInfo("en-us").DateTimeFormat.Clone Dim n As DateTimeFormatInfo = CultureInfo.GetCultureInfo("en-us").DateTimeFormat.Clone

109
SCrawler/API/Twitter/SiteSettings.vb
View File

@@ -12,7 +12,7 @@ Imports SCrawler.Plugin.Attributes
Imports PersonalUtilities.Functions.RegularExpressions Imports PersonalUtilities.Functions.RegularExpressions
Imports PersonalUtilities.Tools.Web.Clients Imports PersonalUtilities.Tools.Web.Clients
Namespace API.Twitter Namespace API.Twitter
<Manifest(TwitterSiteKey), SavedPosts, SpecialForm(False)> <Manifest(TwitterSiteKey), SavedPosts, SeparatedTasks, SpecialForm(False)>
Friend Class SiteSettings : Inherits SiteSettingsBase Friend Class SiteSettings : Inherits SiteSettingsBase
#Region "Token names" #Region "Token names"
Friend Const Header_Authorization As String = "authorization" Friend Const Header_Authorization As String = "authorization"
@@ -41,19 +41,20 @@ Namespace API.Twitter
Return My.Resources.SiteResources.TwitterPic_400 Return My.Resources.SiteResources.TwitterPic_400
End Get End Get
End Property End Property
#Region "Auth" 'TODELETE: twitter headers
<PropertyOption(AllowNull:=False, IsAuth:=True, ControlText:="Authorization", '#Region "Auth"
ControlToolTip:="Set authorization from [authorization] response header. This field must start from [Bearer] key word")> ' <PropertyOption(AllowNull:=False, IsAuth:=False, ControlText:="Authorization",
Private ReadOnly Property Auth As PropertyValue ' ControlToolTip:="Set authorization from [authorization] response header. This field must start from [Bearer] key word")>
<PropertyOption(AllowNull:=False, IsAuth:=True, ControlText:="Token", ControlToolTip:="Set token from [x-csrf-token] response header")> ' Private ReadOnly Property Auth As PropertyValue
Private ReadOnly Property Token As PropertyValue ' <PropertyOption(AllowNull:=False, IsAuth:=False, ControlText:="Token", ControlToolTip:="Set token from [x-csrf-token] response header")>
#End Region ' Private ReadOnly Property Token As PropertyValue
'#End Region
#Region "Other properties" #Region "Other properties"
<PropertyOption(IsAuth:=False, ControlText:=GifsDownload_Text), PXML> <PropertyOption(ControlText:=GifsDownload_Text), PXML>
Friend ReadOnly Property GifsDownload As PropertyValue Friend ReadOnly Property GifsDownload As PropertyValue
<PropertyOption(IsAuth:=False, ControlText:=GifsSpecialFolder_Text, ControlToolTip:=GifsSpecialFolder_ToolTip), PXML> <PropertyOption(ControlText:=GifsSpecialFolder_Text, ControlToolTip:=GifsSpecialFolder_ToolTip), PXML>
Friend ReadOnly Property GifsSpecialFolder As PropertyValue Friend ReadOnly Property GifsSpecialFolder As PropertyValue
<PropertyOption(IsAuth:=False, ControlText:=GifsPrefix_Text, ControlToolTip:=GifsPrefix_ToolTip), PXML> <PropertyOption(ControlText:=GifsPrefix_Text, ControlToolTip:=GifsPrefix_ToolTip), PXML>
Friend ReadOnly Property GifsPrefix As PropertyValue Friend ReadOnly Property GifsPrefix As PropertyValue
<Provider(NameOf(GifsSpecialFolder), Interaction:=True), Provider(NameOf(GifsPrefix), Interaction:=True)> <Provider(NameOf(GifsSpecialFolder), Interaction:=True), Provider(NameOf(GifsPrefix), Interaction:=True)>
Private ReadOnly Property GifStringChecker As IFormatProvider Private ReadOnly Property GifStringChecker As IFormatProvider
@@ -75,68 +76,52 @@ Namespace API.Twitter
Throw New NotImplementedException("[GetFormat] is not available in the context of [GifStringProvider]") Throw New NotImplementedException("[GetFormat] is not available in the context of [GifStringProvider]")
End Function End Function
End Class End Class
<PropertyOption(IsAuth:=False, ControlText:=UseMD5Comparison_Text, ControlToolTip:=UseMD5Comparison_ToolTip), PXML> <PropertyOption(ControlText:=UseMD5Comparison_Text, ControlToolTip:=UseMD5Comparison_ToolTip), PXML>
Friend ReadOnly Property UseMD5Comparison As PropertyValue Friend ReadOnly Property UseMD5Comparison As PropertyValue
<PXML, PropertyOption(ControlText:="Concurrent downloads", ControlToolTip:="The number of concurrent downloads.", LeftOffset:=120), TaskCounter>
Friend ReadOnly Property ConcurrentDownloads As PropertyValue
#End Region #End Region
Friend Overrides ReadOnly Property Responser As Responser 'TODELETE: twitter headers
Private Sub ChangeResponserFields(ByVal PropName As String, ByVal Value As Object) 'Private Sub ChangeResponserFields(ByVal PropName As String, ByVal Value As Object)
If Not PropName.IsEmptyString Then ' If Not PropName.IsEmptyString Then
Dim f$ = String.Empty ' Dim f$ = String.Empty
Select Case PropName ' Select Case PropName
Case NameOf(Auth) : f = Header_Authorization ' Case NameOf(Auth) : f = Header_Authorization
Case NameOf(Token) : f = Header_Token ' Case NameOf(Token) : f = Header_Token
End Select ' End Select
If Not f.IsEmptyString Then ' If Not f.IsEmptyString Then
Responser.Headers.Remove(f) ' Responser.Headers.Remove(f)
If Not CStr(Value).IsEmptyString Then Responser.Headers.Add(f, CStr(Value)) ' If Not CStr(Value).IsEmptyString Then Responser.Headers.Add(f, CStr(Value))
Responser.SaveSettings() ' Responser.SaveSettings()
End If ' End If
End If ' End If
End Sub 'End Sub
#End Region #End Region
Friend Sub New() Friend Sub New()
MyBase.New(TwitterSite) MyBase.New(TwitterSite, "twitter.com")
Responser = New Responser($"{SettingsFolderName}\Responser_{Site}.xml") With {.DeclaredError = EDP.ThrowException}
Dim a$ = String.Empty 'TODELETE: twitter headers
Dim t$ = String.Empty 'Dim a$ = String.Empty
'Dim t$ = String.Empty
With Responser With Responser
If .File.Exists Then 'TODELETE: twitter headers
.CookiesDomain = "twitter.com" 'a = .Headers.Value(Header_Authorization)
.CookiesEncryptKey = SettingsCLS.CookieEncryptKey 't = .Headers.Value(Header_Token)
.LoadSettings()
a = .Headers.Value(Header_Authorization)
t = .Headers.Value(Header_Token)
Else
.ContentType = "application/json"
.Accept = "*/*"
.CookiesDomain = "twitter.com"
.CookiesEncryptKey = SettingsCLS.CookieEncryptKey
.Decoders.Add(SymbolsConverter.Converters.Unicode)
.Headers.Add("sec-ch-ua", """Chromium"";v=""112"", ""Google Chrome"";v=""112"", ""Not:A-Brand"";v=""99""")
.Headers.Add("sec-ch-ua-mobile", "?0")
.Headers.Add("sec-fetch-dest", "empty")
.Headers.Add("sec-fetch-mode", "cors")
.Headers.Add("sec-fetch-site", "same-origin")
.Headers.Add(Header_Token, String.Empty)
.Headers.Add("x-twitter-active-user", "yes")
.Headers.Add("x-twitter-auth-type", "OAuth2Session")
.Headers.Add(Header_Authorization, String.Empty)
.SaveSettings()
End If
.Cookies.ChangedAllowInternalDrop = False .Cookies.ChangedAllowInternalDrop = False
.Cookies.Changed = False .Cookies.Changed = False
End With End With
Auth = New PropertyValue(a, GetType(String), Sub(v) ChangeResponserFields(NameOf(Auth), v)) 'TODELETE: twitter headers
Token = New PropertyValue(t, GetType(String), Sub(v) ChangeResponserFields(NameOf(Token), v)) 'Auth = New PropertyValue(a, GetType(String), Sub(v) ChangeResponserFields(NameOf(Auth), v))
'Token = New PropertyValue(t, GetType(String), Sub(v) ChangeResponserFields(NameOf(Token), v))
GifsDownload = New PropertyValue(True) GifsDownload = New PropertyValue(True)
GifsSpecialFolder = New PropertyValue(String.Empty, GetType(String)) GifsSpecialFolder = New PropertyValue(String.Empty, GetType(String))
GifsPrefix = New PropertyValue("GIF_") GifsPrefix = New PropertyValue("GIF_")
GifStringChecker = New GifStringProvider GifStringChecker = New GifStringProvider
UseMD5Comparison = New PropertyValue(False) UseMD5Comparison = New PropertyValue(False)
ConcurrentDownloads = New PropertyValue(1)
UserRegex = RParams.DMS("[htps:/]{7,8}.*?twitter.com/([^/]+)", 1) UserRegex = RParams.DMS("[htps:/]{7,8}.*?twitter.com/([^/]+)", 1)
UrlPatternUser = "https://twitter.com/{0}" UrlPatternUser = "https://twitter.com/{0}"
@@ -151,18 +136,10 @@ Namespace API.Twitter
Return $"https://twitter.com/{User.Name}/status/{Media.Post.ID}" Return $"https://twitter.com/{User.Name}/status/{Media.Post.ID}"
End Function End Function
Friend Overrides Function BaseAuthExists() As Boolean Friend Overrides Function BaseAuthExists() As Boolean
Return Responser.CookiesExists And ACheck(Token.Value) And ACheck(Auth.Value) Return Responser.CookiesExists
End Function End Function
Friend Overrides Function Available(ByVal What As ISiteSettings.Download, ByVal Silent As Boolean) As Boolean Friend Overrides Function Available(ByVal What As ISiteSettings.Download, ByVal Silent As Boolean) As Boolean
If MyBase.Available(What, Silent) Then Return Settings.GalleryDLFile.Exists And BaseAuthExists()
If What = ISiteSettings.Download.SavedPosts Then
Return Settings.GalleryDLFile.Exists
Else
Return True
End If
Else
Return False
End If
End Function End Function
Friend Overrides Sub UserOptions(ByRef Options As Object, ByVal OpenForm As Boolean) Friend Overrides Sub UserOptions(ByRef Options As Object, ByVal OpenForm As Boolean)
If Options Is Nothing OrElse (Not TypeOf Options Is EditorExchangeOptions OrElse If Options Is Nothing OrElse (Not TypeOf Options Is EditorExchangeOptions OrElse

341
SCrawler/API/Twitter/UserData.vb
View File

@@ -6,19 +6,17 @@
' '
' This program is distributed in the hope that it will be useful, ' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY ' but WITHOUT ANY WARRANTY
Imports System.Net
Imports System.Threading Imports System.Threading
Imports SCrawler.API.Base Imports SCrawler.API.Base
Imports SCrawler.API.YouTube.Objects Imports SCrawler.API.YouTube.Objects
Imports PersonalUtilities.Functions.XML Imports PersonalUtilities.Functions.XML
Imports PersonalUtilities.Functions.RegularExpressions Imports PersonalUtilities.Functions.RegularExpressions
Imports PersonalUtilities.Tools.Web.Clients Imports PersonalUtilities.Tools
Imports PersonalUtilities.Tools.Web.Documents.JSON Imports PersonalUtilities.Tools.Web.Documents.JSON
Imports UStates = SCrawler.API.Base.UserMedia.States Imports UStates = SCrawler.API.Base.UserMedia.States
Imports UTypes = SCrawler.API.Base.UserMedia.Types Imports UTypes = SCrawler.API.Base.UserMedia.Types
Namespace API.Twitter Namespace API.Twitter
Friend Class UserData : Inherits UserDataBase Friend Class UserData : Inherits UserDataBase
Protected SinglePostUrl As String = "https://api.twitter.com/1.1/statuses/show.json?id={0}&tweet_mode=extended"
#Region "XML names" #Region "XML names"
Private Const Name_GifsDownload As String = "GifsDownload" Private Const Name_GifsDownload As String = "GifsDownload"
Private Const Name_GifsSpecialFolder As String = "GifsSpecialFolder" Private Const Name_GifsSpecialFolder As String = "GifsSpecialFolder"
@@ -29,6 +27,19 @@ Namespace API.Twitter
Friend Property GifsSpecialFolder As String = String.Empty Friend Property GifsSpecialFolder As String = String.Empty
Friend Property GifsPrefix As String = String.Empty Friend Property GifsPrefix As String = String.Empty
Private ReadOnly _DataNames As List(Of String) Private ReadOnly _DataNames As List(Of String)
Private ReadOnly Property MySettings As SiteSettings
Get
Return HOST.Source
End Get
End Property
Private FileNameProvider As ANumbers = Nothing
Private Sub ResetFileNameProvider(Optional ByVal GroupSize As Integer? = Nothing)
FileNameProvider = New ANumbers With {.FormatOptions = ANumbers.Options.FormatNumberGroup + ANumbers.Options.Groups}
FileNameProvider.GroupSize = If(GroupSize, 3)
End Sub
Private Function RenameGdlFile(ByVal Input As SFile, ByVal i As Integer) As SFile
Return SFile.Rename(Input, $"{Input.PathWithSeparator}{i.NumToString(FileNameProvider)}.{Input.Extension}",, EDP.ThrowException)
End Function
#End Region #End Region
#Region "Exchange options" #Region "Exchange options"
Friend Overrides Function ExchangeOptionsGet() As Object Friend Overrides Function ExchangeOptionsGet() As Object
@@ -79,31 +90,55 @@ Namespace API.Twitter
DownloadData_SavedPosts(Token) DownloadData_SavedPosts(Token)
Else Else
If _ContentList.Count > 0 Then _DataNames.ListAddList(_ContentList.Select(Function(c) c.File.File), LAP.ClearBeforeAdd, LAP.NotContainsOnly) If _ContentList.Count > 0 Then _DataNames.ListAddList(_ContentList.Select(Function(c) c.File.File), LAP.ClearBeforeAdd, LAP.NotContainsOnly)
DownloadData(String.Empty, Token) DownloadData_Timeline(Token)
End If End If
End Sub End Sub
Private Overloads Sub DownloadData(ByVal POST As String, ByVal Token As CancellationToken) Private Sub DownloadData_Timeline(ByVal Token As CancellationToken)
Dim URL$ = String.Empty Dim URL$ = String.Empty
Dim tCache As CacheKeeper = Nothing
Try Try
Dim PostID$ = String.Empty Dim PostID$ = String.Empty
Dim PostDate$ Dim PostDate$, tmpUserId$
Dim j As EContainer
Dim nn As EContainer Dim nn As EContainer
Dim NewPostDetected As Boolean = False Dim NewPostDetected As Boolean = False
Dim ExistsDetected As Boolean = False Dim ExistsDetected As Boolean = False
Dim UID As Func(Of EContainer, String) = Function(e) e.XmlIfNothing.Item({"user", "id"}).XmlIfNothingValue tCache = New CacheKeeper($"{DownloadContentDefault_GetRootDir()}\_tCache\")
If tCache.RootDirectory.Exists(SFO.Path, False) Then tCache.RootDirectory.Delete(SFO.Path, SFODelete.DeletePermanently, EDP.ReturnValue)
URL = $"https://api.twitter.com/1.1/statuses/user_timeline.json?screen_name={Name}&count=200&exclude_replies=false&include_rts=1&tweet_mode=extended" tCache.Validate()
If Not POST.IsEmptyString Then URL &= $"&max_id={POST}" Dim f As SFile = GetTimelineFromGalleryDL(tCache.RootDirectory, Token)
If Not f.IsEmptyString Then
ThrowAny(Token) ThrowAny(Token)
Dim r$ = Responser.GetResponse(URL) Dim timelineFiles As List(Of SFile) = SFile.GetFiles(f, "*.txt",, EDP.ReturnValue)
If Not r.IsEmptyString Then If timelineFiles.ListExists Then
Using w As EContainer = JsonDocument.Parse(r) Dim i%
If w.ListExists Then ResetFileNameProvider(Math.Max(timelineFiles.Count.ToString.Length, 2))
'rename files
If POST.IsEmptyString And Not w.ItemF({0, "user"}) Is Nothing Then For i = 0 To timelineFiles.Count - 1 : timelineFiles(i) = RenameGdlFile(timelineFiles(i), i) : Next
With w.ItemF({0, "user"}) 'parse files
For i = 0 To timelineFiles.Count - 1
j = JsonDocument.Parse(timelineFiles(i).GetText)
If Not j Is Nothing Then
If i = 0 Then
Dim resValue$ = j.Value({"data", "user", "result"}, "__typename").StringTrim.StringToLower
If resValue.IsEmptyString Then
UserExists = False
j.Dispose()
Exit Sub
ElseIf resValue = "userunavailable" Then
UserSuspended = True
j.Dispose()
Exit Sub
Else
With j({"data", "user", "result"})
If .ListExists Then
If ID.IsEmptyString Then
ID = .Value("rest_id")
If Not ID.IsEmptyString Then _ForceSaveUserInfo = True
End If
With .Item({"legacy"})
If .ListExists Then
If .Value("screen_name").StringToLower = Name.ToLower Then If .Value("screen_name").StringToLower = Name.ToLower Then
UserSiteNameUpdate(.Value("name")) UserSiteNameUpdate(.Value("name"))
UserDescriptionUpdate(.Value("description")) UserDescriptionUpdate(.Value("description"))
@@ -114,28 +149,30 @@ Namespace API.Twitter
If __imgFile.Extension.IsEmptyString Then __imgFile.Extension = "jpg" If __imgFile.Extension.IsEmptyString Then __imgFile.Extension = "jpg"
__imgFile.Path = MyFile.CutPath.Path __imgFile.Path = MyFile.CutPath.Path
If Not __imgFile.Exists Then GetWebFile(img, __imgFile, EDP.None) If Not __imgFile.Exists Then GetWebFile(img, __imgFile, EDP.None)
If __imgFile.Exists Then IconBannerDownloaded = True
End If End If
End If End If
End Sub End Sub
Dim icon$ = .Value("profile_image_url_https") Dim icon$ = .Value("profile_image_url_https")
If Not icon.IsEmptyString Then icon = icon.Replace("_normal", String.Empty) If Not icon.IsEmptyString Then icon = icon.Replace("_normal", String.Empty)
If DownloadIconBanner Then
__getImage.Invoke(.Value("profile_banner_url")) __getImage.Invoke(.Value("profile_banner_url"))
__getImage.Invoke(icon) __getImage.Invoke(icon)
End If End If
End If
End If
End With End With
End If End If
End With
With If(IsSavedPosts, w({"globalObjects", "tweets"}).XmlIfNothing, w) End If
Else
With j({"globalObjects", "tweets"})
If .ListExists Then
ProgressPre.ChangeMax(.Count) ProgressPre.ChangeMax(.Count)
For Each nn In .Self For Each nn In .Self
ProgressPre.Perform() ProgressPre.Perform()
ThrowAny(Token)
If nn.Count > 0 Then If nn.Count > 0 Then
PostID = nn.Value("id") PostID = nn.Value("id")
If ID.IsEmptyString Then
ID = UID(nn)
If Not ID.IsEmptyString Then UpdateUserInformation()
End If
'Date Pattern: 'Date Pattern:
'Sat Jan 01 01:10:15 +0000 2000 'Sat Jan 01 01:10:15 +0000 2000
@@ -153,60 +190,57 @@ Namespace API.Twitter
Continue For Continue For
End If End If
If Not ParseUserMediaOnly OrElse tmpUserId = nn.ItemF({"extended_entities", "media", 0, "source_user_id"}).
(Not nn.Contains("retweeted_status") OrElse (Not ID.IsEmptyString AndAlso UID(nn("retweeted_status")) = ID)) Then _ XmlIfNothingValue.IfNullOrEmpty(nn.Value("user_id")).IfNullOrEmpty("/")
If Not ParseUserMediaOnly OrElse (Not ID.IsEmptyString AndAlso tmpUserId = ID) Then _
ObtainMedia(nn, PostID, PostDate) ObtainMedia(nn, PostID, PostDate)
End If End If
Next Next
End If
End With End With
End If End If
End Using j.Dispose()
End If
If POST.IsEmptyString And ExistsDetected Then Exit Sub Next
If Not PostID.IsEmptyString And NewPostDetected Then DownloadData(PostID, Token) End If
End If End If
Catch ex As Exception Catch ex As Exception
ProcessException(ex, Token, $"data downloading error [{URL}]") ProcessException(ex, Token, $"data downloading error [{URL}]")
Finally
If Not tCache Is Nothing Then tCache.Dispose()
End Try End Try
End Sub End Sub
Private Sub DownloadData_SavedPosts(ByVal Token As CancellationToken) Private Sub DownloadData_SavedPosts(ByVal Token As CancellationToken)
Try Try
Dim urls As List(Of String) = GetBookmarksUrlsFromGalleryDL() Dim f As SFile = GetDataFromGalleryDL("https://twitter.com/i/bookmarks", Settings.Cache, True, Token)
If urls.ListExists Then Dim files As List(Of SFile) = SFile.GetFiles(f, "*.txt")
Dim postIds As New List(Of String) If files.ListExists Then
Dim r$ ResetFileNameProvider(Math.Max(files.Count.ToString.Length, 3))
Dim id$
Dim j As EContainer, jj As EContainer Dim j As EContainer, jj As EContainer
Dim jErr As New ErrorsDescriber(EDP.ReturnValue) Dim jErr As New ErrorsDescriber(EDP.ReturnValue)
Dim rPattern As RParams = RParams.DM("(?<=tweet-)(\d+)\Z", 0, EDP.ReturnValue) For i% = 0 To files.Count - 1
ProgressPre.ChangeMax(urls.Count) f = RenameGdlFile(files(i), i)
For Each url$ In urls j = JsonDocument.Parse(f.GetText, jErr)
ProgressPre.Perform()
r = Responser.GetResponse(url)
If Not r.IsEmptyString Then
j = JsonDocument.Parse(r, jErr)
If Not j Is Nothing Then If Not j Is Nothing Then
jj = j.ItemF({"data", "bookmark_timeline_v2", "timeline", "instructions", 0, "entries"}) With j.ItemF({"data", 0, "timeline", "instructions", 0, "entries"})
If If(jj?.Count, 0) > 0 Then postIds.ListAddList(jj.Select(Function(jj2) CStr(RegexReplace(jj2.Value("entryId"), rPattern))), LNC) If .ListExists Then
j.Dispose() ProgressPre.ChangeMax(.Count)
End If For Each jj In .Self
End If
Next
If postIds.Count > 0 Then postIds.RemoveAll(Function(pid) pid.IsEmptyString OrElse (_TempPostsList.Contains(pid) Or _DataNames.Contains(pid)))
If postIds.Count > 0 Then
ProgressPre.ChangeMax(postIds.Count)
For Each __id$ In postIds
ProgressPre.Perform() ProgressPre.Perform()
_TempPostsList.Add(__id) With jj({"content", "itemContent", "tweet_results", "result", "legacy"})
r = Responser.GetResponse(String.Format(SinglePostUrl, __id),, EDP.ReturnValue) If .ListExists Then
If Not r.IsEmptyString Then id = .Value("id_str")
j = JsonDocument.Parse(r, jErr) If _TempPostsList.Contains(id) Then j.Dispose() : Exit Sub Else ObtainMedia(.Self, id, .Value("created_at"))
If Not j Is Nothing Then
If j.Count > 0 Then ObtainMedia(j, __id, j.Value("created_at"))
j.Dispose()
End If
End If End If
End With
Next Next
End If End If
End With
j.Dispose()
End If
Next
End If End If
Catch ex As Exception Catch ex As Exception
ProcessException(ex, Token, "data downloading error (Saved Posts)") ProcessException(ex, Token, "data downloading error (Saved Posts)")
@@ -215,22 +249,23 @@ Namespace API.Twitter
#End Region #End Region
#Region "Obtain media" #Region "Obtain media"
Private Sub ObtainMedia(ByVal e As EContainer, ByVal PostID As String, ByVal PostDate As String, Optional ByVal State As UStates = UStates.Unknown) Private Sub ObtainMedia(ByVal e As EContainer, ByVal PostID As String, ByVal PostDate As String, Optional ByVal State As UStates = UStates.Unknown)
If Not CheckVideoNode(e, PostID, PostDate, State) Then
Dim s As EContainer = e.ItemF({"extended_entities", "media"}) Dim s As EContainer = e.ItemF({"extended_entities", "media"})
If s Is Nothing OrElse s.Count = 0 Then s = e.ItemF({"retweeted_status", "extended_entities", "media"}) If s Is Nothing OrElse s.Count = 0 Then s = e.ItemF({"retweeted_status", "extended_entities", "media"})
If If(s?.Count, 0) > 0 Then If If(s?.Count, 0) > 0 Then
For Each m In s Dim mUrl$
If m.Contains("media_url") Then For Each m As EContainer In s
Dim dName$ = UrlFile(m("media_url").Value) If Not CheckVideoNode(m, PostID, PostDate, State) Then
mUrl = m.Value("media_url").IfNullOrEmpty(m.Value("media_url_https"))
If Not mUrl.IsEmptyString Then
Dim dName$ = UrlFile(mUrl)
If Not dName.IsEmptyString AndAlso Not _DataNames.Contains(dName) Then If Not dName.IsEmptyString AndAlso Not _DataNames.Contains(dName) Then
_DataNames.Add(dName) _DataNames.Add(dName)
_TempMediaList.ListAddValue(MediaFromData(m("media_url").Value, _TempMediaList.ListAddValue(MediaFromData(mUrl, PostID, PostDate, GetPictureOption(m), State, UTypes.Picture), LNC)
PostID, PostDate, GetPictureOption(m), State, UTypes.Picture), LNC) End If
End If End If
End If End If
Next Next
End If End If
End If
End Sub End Sub
Private Function CheckVideoNode(ByVal w As EContainer, ByVal PostID As String, ByVal PostDate As String, Private Function CheckVideoNode(ByVal w As EContainer, ByVal PostID As String, ByVal PostDate As String,
Optional ByVal State As UStates = UStates.Unknown) As Boolean Optional ByVal State As UStates = UStates.Unknown) As Boolean
@@ -260,9 +295,8 @@ Namespace API.Twitter
Dim url$, ff$ Dim url$, ff$
Dim f As SFile Dim f As SFile
Dim m As UserMedia Dim m As UserMedia
With w({"extended_entities", "media"}) If w.ListExists Then
If .ListExists Then For Each n As EContainer In w
For Each n As EContainer In .Self
If n.Value("type") = "animated_gif" Then If n.Value("type") = "animated_gif" Then
With n({"video_info", "variants"}) With n({"video_info", "variants"})
If .ListExists Then If .ListExists Then
@@ -287,7 +321,6 @@ Namespace API.Twitter
End If End If
Next Next
End If End If
End With
Return False Return False
Catch ex As Exception Catch ex As Exception
LogError(ex, "[API.Twitter.UserData.CheckForGif]") LogError(ex, "[API.Twitter.UserData.CheckForGif]")
@@ -295,65 +328,154 @@ Namespace API.Twitter
End Try End Try
End Function End Function
Private Function GetVideoNodeURL(ByVal w As EContainer) As String Private Function GetVideoNodeURL(ByVal w As EContainer) As String
Dim v As EContainer = w.GetNode(VideoNode) With w({"video_info", "variants"})
If v.ListExists Then If .ListExists Then
Dim l As New List(Of Sizes) Dim l As New List(Of Sizes)
Dim u$ Dim u$
Dim nn As EContainer For Each n As EContainer In .Self
For Each n As EContainer In v
If n.Count > 0 Then If n.Count > 0 Then
For Each nn In n If n("content_type").XmlIfNothingValue("none").Contains("mp4") AndAlso n.Contains("url") Then
If nn("content_type").XmlIfNothingValue("none").Contains("mp4") AndAlso nn.Contains("url") Then u = n.Value("url")
u = nn.Value("url")
l.Add(New Sizes(RegexReplace(u, VideoSizeRegEx), u)) l.Add(New Sizes(RegexReplace(u, VideoSizeRegEx), u))
End If End If
Next
End If End If
Next Next
If l.Count > 0 Then l.RemoveAll(Function(s) s.HasError) If l.Count > 0 Then l.RemoveAll(Function(s) s.HasError)
If l.Count > 0 Then l.Sort() : Return l(0).Data If l.Count > 0 Then l.Sort() : Return l(0).Data
End If End If
End With
Return String.Empty Return String.Empty
End Function End Function
#End Region #End Region
#Region "Gallery-DL Support" #Region "Gallery-DL Support"
Private Function GetBookmarksUrlsFromGalleryDL() As List(Of String) Private Class TwitterGDL : Inherits GDL.GDLBatch
Dim command$ = $"gallery-dl --verbose --simulate --cookies ""{DirectCast(HOST.Source, SiteSettings).CookiesNetscapeFile}"" https://twitter.com/i/bookmarks" Private Property Token As CancellationToken
Friend Sub New(ByVal Dir As SFile, ByVal _Token As CancellationToken)
MyBase.New
Commands.Clear()
ChangeDirectory(Dir)
Token = _Token
End Sub
Protected Overrides Async Function Validate(ByVal Value As String) As Task
If Not ProcessKilled AndAlso Await Task.Run(Function() Token.IsCancellationRequested OrElse IdExists(Value)) Then Kill()
End Function
Private Function IdExists(ByVal Value As String) As Boolean
Try Try
Using batch As New GDL.GDLBatch With {.TempPostsList = _TempPostsList} : Return GDL.GetUrlsFromGalleryDl(batch, command) : End Using Value = Value.StringTrim
If Not Value.IsEmptyString AndAlso (Value.StartsWith("*") Or Value.StartsWith(".\gallery-dl\")) Then
Dim id$ = Value.Split("\").Last.Split(".").First.Split("_").First
If Not id.IsEmptyString Then Return TempPostsList.Contains(id)
End If
Catch ex As Exception Catch ex As Exception
HasError = True
LogError(ex, $"GetJson({command})")
Return Nothing
End Try End Try
Return False
End Function
End Class
Private Function GetDataFromGalleryDL(ByVal URL As String, ByVal Cache As CacheKeeper, ByVal UseTempPostList As Boolean,
Optional ByVal Token As CancellationToken = Nothing) As SFile
Dim command$ = $"""{Settings.GalleryDLFile}"" --verbose --no-download --no-skip --cookies ""{MySettings.CookiesNetscapeFile}"" --write-pages "
Try
Dim dir As SFile = Cache.NewPath
If dir.Exists(SFO.Path,, EDP.ThrowException) Then
Using batch As New TwitterGDL(dir, Token)
If UseTempPostList Then
batch.TempPostsList = _TempPostsList
command &= GdlGetIdFilterString()
End If
command &= URL
'#If DEBUG Then
' Debug.WriteLine(command)
'#End If
batch.Execute(command)
End Using
Return dir
End If
Return Nothing
Catch ex As Exception
Return ErrorsDescriber.Execute(EDP.SendToLog, ex, $"{ToStringForLog()}: GetDataFromGalleryDL({command})")
End Try
End Function
Private Function GetTimelineFromGalleryDL(ByVal Cache As CacheKeeper, ByVal Token As CancellationToken) As SFile
Dim command$ = String.Empty
Try
Dim conf As SFile = $"{Cache.NewPath.PathWithSeparator}TwitterGdlConfig.conf"
Dim confText$ = "{""extractor"":{""cookies"": """ & MySettings.CookiesNetscapeFile.ToString.Replace("\", "/") &
""",""cookies-update"": false,""twitter"":{""cards"": false,""conversations"": false,""pinned"": false,""quoted"": false,""replies"": true,""retweets"": true,""strategy"": null,""text-tweets"": false,""twitpic"": false,""unique"": true,""users"": ""timeline"",""videos"": true}}}"
If conf.Exists(SFO.Path, True, EDP.ThrowException) Then TextSaver.SaveTextToFile(confText, conf)
If Not conf.Exists Then Throw New IO.FileNotFoundException("Can't find Twitter GDL config file", conf)
command = $"""{Settings.GalleryDLFile}"" --verbose --no-download --no-skip --config ""{conf}"" --write-pages "
command &= GdlGetIdFilterString()
command &= $"https://twitter.com/search?q=from:{Name}+include:nativeretweets"
Dim dir As SFile = Cache.NewPath
dir.Exists(SFO.Path, True, EDP.ThrowException)
'#If DEBUG Then
' Debug.WriteLine(command)
'#End If
Using tgdl As New TwitterGDL(dir, Token) With {.TempPostsList = _TempPostsList} : tgdl.Execute(command) : End Using
Return dir
Catch ex As Exception
Return ErrorsDescriber.Execute(EDP.SendToLog, ex, $"{ToStringForLog()}: GetTimelineFromGalleryDL({command})")
End Try
End Function
Private Function GdlGetIdFilterString() As String
Return If(_TempPostsList.Count > 0, $"--filter ""int(tweet_id) > {_TempPostsList.Last} or abort()"" ", String.Empty)
End Function End Function
#End Region #End Region
#Region "ReparseMissing" #Region "ReparseMissing"
Protected Overrides Sub ReparseMissing(ByVal Token As CancellationToken) Protected Overrides Sub ReparseMissing(ByVal Token As CancellationToken)
Const SinglePostPattern$ = "https://twitter.com/{0}/status/{1}"
Dim rList As New List(Of Integer) Dim rList As New List(Of Integer)
Dim URL$ = String.Empty Dim URL$ = String.Empty
Dim cache As CacheKeeper = Nothing
Try Try
If ContentMissingExists Then If ContentMissingExists Then
Dim m As UserMedia Dim m As UserMedia
Dim r$, PostDate$ Dim PostDate$
Dim j As EContainer Dim j As EContainer
Dim f As SFile
Dim i%, ii%
Dim files As List(Of SFile)
ResetFileNameProvider()
If IsSingleObjectDownload Then
cache = Settings.Cache
Else
cache = New CacheKeeper(DownloadContentDefault_GetRootDir.CSFilePS)
End If
ProgressPre.ChangeMax(_ContentList.Count) ProgressPre.ChangeMax(_ContentList.Count)
For i% = 0 To _ContentList.Count - 1 For i = 0 To _ContentList.Count - 1
ProgressPre.Perform() ProgressPre.Perform()
If _ContentList(i).State = UStates.Missing Then If _ContentList(i).State = UStates.Missing Then
m = _ContentList(i) m = _ContentList(i)
If Not m.Post.ID.IsEmptyString Then If Not m.Post.ID.IsEmptyString Or (IsSingleObjectDownload And Not m.URL_BASE.IsEmptyString) Then
ThrowAny(Token) ThrowAny(Token)
URL = String.Format(SinglePostUrl, m.Post.ID) If IsSingleObjectDownload Then
r = Responser.GetResponse(URL,, EDP.ReturnValue) URL = m.URL_BASE
If Not r.IsEmptyString Then Else
j = JsonDocument.Parse(r) URL = String.Format(SinglePostPattern, Name, m.Post.ID)
End If
f = GetDataFromGalleryDL(URL, cache, Favorite, Token)
If Not f.IsEmptyString Then
files = SFile.GetFiles(f, "*.txt")
If files.ListExists Then
For ii = 0 To files.Count - 1
f = RenameGdlFile(files(ii), ii)
j = JsonDocument.Parse(f.GetText)
If Not j Is Nothing Then If Not j Is Nothing Then
With j.ItemF({"data", 0, "instructions", 0, "entries", 0,
"content", "itemContent", "tweet_results", "result", "legacy"})
If .ListExists Then
PostDate = String.Empty PostDate = String.Empty
If j.Contains("created_at") Then PostDate = j("created_at").Value Else PostDate = String.Empty If .Contains("created_at") Then PostDate = .Value("created_at") Else PostDate = String.Empty
ObtainMedia(j, m.Post.ID, PostDate, UStates.Missing) ObtainMedia(.Self, m.Post.ID, PostDate, UStates.Missing)
rList.Add(i) rList.Add(i)
End If End If
End With
j.Dispose()
End If
Next
files.Clear()
End If
End If End If
End If End If
End If End If
@@ -362,6 +484,7 @@ Namespace API.Twitter
Catch ex As Exception Catch ex As Exception
ProcessException(ex, Token, $"ReparseMissing error [{URL}]") ProcessException(ex, Token, $"ReparseMissing error [{URL}]")
Finally Finally
If Not cache Is Nothing And Not IsSingleObjectDownload Then cache.Dispose()
If rList.Count > 0 Then If rList.Count > 0 Then
For i% = rList.Count - 1 To 0 Step -1 : _ContentList.RemoveAt(i) : Next For i% = rList.Count - 1 To 0 Step -1 : _ContentList.RemoveAt(i) : Next
rList.Clear() rList.Clear()
@@ -371,15 +494,8 @@ Namespace API.Twitter
#End Region #End Region
#Region "DownloadSingleObject" #Region "DownloadSingleObject"
Protected Overrides Sub DownloadSingleObject_GetPosts(ByVal Data As IYouTubeMediaContainer, ByVal Token As CancellationToken) Protected Overrides Sub DownloadSingleObject_GetPosts(ByVal Data As IYouTubeMediaContainer, ByVal Token As CancellationToken)
Dim PostID$ = RegexReplace(Data.URL, RParams.DM("(?<=/)\d+", 0)) _ContentList.Add(New UserMedia(Data.URL) With {.State = UStates.Missing})
If Not PostID.IsEmptyString Then ReparseMissing(Token)
Dim r$ = Responser.GetResponse(String.Format(SinglePostUrl, PostID),, EDP.ReturnValue)
If Not r.IsEmptyString Then
Using j As EContainer = JsonDocument.Parse(r)
If j.ListExists Then ObtainMedia(j, j.Value("id"), j.Value("created_at"))
End Using
End If
End If
End Sub End Sub
#End Region #End Region
#Region "Picture options" #Region "Picture options"
@@ -459,26 +575,7 @@ Namespace API.Twitter
#Region "Exception" #Region "Exception"
Protected Overrides Function DownloadingException(ByVal ex As Exception, ByVal Message As String, Optional ByVal FromPE As Boolean = False, Protected Overrides Function DownloadingException(ByVal ex As Exception, ByVal Message As String, Optional ByVal FromPE As Boolean = False,
Optional ByVal EObj As Object = Nothing) As Integer Optional ByVal EObj As Object = Nothing) As Integer
If AEquals(EObj, VALIDATE_MD5_ERROR) Then
If Not FromPE Then LogError(ex, Message)
Return 0 Return 0
Else
With Responser
If .StatusCode = HttpStatusCode.NotFound Then
UserExists = False
ElseIf .StatusCode = HttpStatusCode.Unauthorized Then
UserSuspended = True
ElseIf .StatusCode = HttpStatusCode.BadRequest Then
MyMainLOG = "Twitter has invalid credentials"
ElseIf .StatusCode = HttpStatusCode.ServiceUnavailable Or .StatusCode = HttpStatusCode.InternalServerError Then
MyMainLOG = $"[{CInt(.StatusCode)}] Twitter is currently unavailable ({ToString()})"
Else
If Not FromPE Then LogError(ex, Message) : HasError = True
Return 0
End If
End With
End If
Return 1
End Function End Function
#End Region #End Region
#Region "IDisposable support" #Region "IDisposable support"

2
SCrawler/Download/DownloadProgress.vb
View File

@@ -196,8 +196,10 @@ Namespace DownloadObjects
End If End If
End Sub End Sub
Private Sub JobProgress_Progress0Changed(ByVal Sender As Object, ByVal e As ProgressEventArgs) Private Sub JobProgress_Progress0Changed(ByVal Sender As Object, ByVal e As ProgressEventArgs)
If Not Job.Type = Download.SavedPosts Then
MainProgress.Value0 = DirectCast(Sender, MyProgressExt).Value0 MainProgress.Value0 = DirectCast(Sender, MyProgressExt).Value0
MainProgress.Perform0(0) MainProgress.Perform0(0)
End If
End Sub End Sub
#End Region #End Region
#Region "IDisposable Support" #Region "IDisposable Support"

5
SCrawler/Download/STDownloader/VideoDownloaderForm.vb
View File

@@ -43,7 +43,7 @@ Namespace DownloadObjects.STDownloader
Return ErrorsDescriber.Execute(EDP.SendToLog, ex, "VideoListForm.LoadData_GetFiles", New List(Of IYouTubeMediaContainer)) Return ErrorsDescriber.Execute(EDP.SendToLog, ex, "VideoListForm.LoadData_GetFiles", New List(Of IYouTubeMediaContainer))
End Try End Try
End Function End Function
Protected Overrides Sub BTT_ADD_KeyClick(ByVal Sender As ToolStripMenuItemKeyClick, ByVal e As KeyClickEventArgs) Protected Overrides Sub BTT_ADD_KeyClick(ByVal Sender As ToolStripMenuItemKeyClick, ByVal e As KeyClickEventArgs) Handles BTT_ADD_URLS_ARR.KeyClick
Dim __tag$ = UniversalFunctions.IfNullOrEmpty(Of Object)(Sender.Tag, String.Empty) Dim __tag$ = UniversalFunctions.IfNullOrEmpty(Of Object)(Sender.Tag, String.Empty)
If Not __tag = "a" And Not __tag = UrlsArrTag Then If Not __tag = "a" And Not __tag = UrlsArrTag Then
MyBase.BTT_ADD_KeyClick(Sender, e) MyBase.BTT_ADD_KeyClick(Sender, e)
@@ -51,6 +51,7 @@ Namespace DownloadObjects.STDownloader
Dim url$ = String.Empty Dim url$ = String.Empty
Try Try
url = BufferText url = BufferText
Dim disableDown As Boolean = e.Shift
Dim output As SFile = Settings.LatestSavingPath Dim output As SFile = Settings.LatestSavingPath
Dim isArr As Boolean = __tag = UrlsArrTag Dim isArr As Boolean = __tag = UrlsArrTag
Dim formOpened As Boolean = False Dim formOpened As Boolean = False
@@ -152,7 +153,7 @@ Namespace DownloadObjects.STDownloader
If media Is Nothing Then If media Is Nothing Then
MsgBoxE({$"The URL you entered is not recognized by existing plugins.{vbCr}{url}", "Download video"}, vbCritical) MsgBoxE({$"The URL you entered is not recognized by existing plugins.{vbCr}{url}", "Download video"}, vbCritical)
Else Else
ControlCreateAndAdd(media) ControlCreateAndAdd(media, disableDown)
End If End If
End If End If
Catch ex As Exception Catch ex As Exception

111
SCrawler/Editors/GlobalSettingsForm.Designer.vb generated
View File

@@ -122,6 +122,8 @@ Namespace Editors
Me.CH_NAME_SITE_FRIENDLY = New System.Windows.Forms.CheckBox() Me.CH_NAME_SITE_FRIENDLY = New System.Windows.Forms.CheckBox()
Me.CH_STD = New System.Windows.Forms.CheckBox() Me.CH_STD = New System.Windows.Forms.CheckBox()
Me.CH_STD_EVERY = New System.Windows.Forms.CheckBox() Me.CH_STD_EVERY = New System.Windows.Forms.CheckBox()
Me.CH_STD_YT_LOAD = New System.Windows.Forms.CheckBox()
Me.CH_STD_YT_REMOVE = New System.Windows.Forms.CheckBox()
Me.TXT_CHANNELS_ROWS = New PersonalUtilities.Forms.Controls.TextBoxExtended() Me.TXT_CHANNELS_ROWS = New PersonalUtilities.Forms.Controls.TextBoxExtended()
Me.TXT_CHANNELS_COLUMNS = New PersonalUtilities.Forms.Controls.TextBoxExtended() Me.TXT_CHANNELS_COLUMNS = New PersonalUtilities.Forms.Controls.TextBoxExtended()
Me.CH_DOWN_IMAGES_NATIVE = New System.Windows.Forms.CheckBox() Me.CH_DOWN_IMAGES_NATIVE = New System.Windows.Forms.CheckBox()
@@ -157,8 +159,7 @@ Namespace Editors
Me.TAB_MAIN = New System.Windows.Forms.TabControl() Me.TAB_MAIN = New System.Windows.Forms.TabControl()
Me.TAB_ENVIR = New System.Windows.Forms.TabPage() Me.TAB_ENVIR = New System.Windows.Forms.TabPage()
Me.CONTAINER_MAIN = New System.Windows.Forms.ToolStripContainer() Me.CONTAINER_MAIN = New System.Windows.Forms.ToolStripContainer()
Me.CH_STD_YT_LOAD = New System.Windows.Forms.CheckBox() Me.CH_UICON_UP = New System.Windows.Forms.CheckBox()
Me.CH_STD_YT_REMOVE = New System.Windows.Forms.CheckBox()
TP_BASIS = New System.Windows.Forms.TableLayoutPanel() TP_BASIS = New System.Windows.Forms.TableLayoutPanel()
TP_IMAGES = New System.Windows.Forms.TableLayoutPanel() TP_IMAGES = New System.Windows.Forms.TableLayoutPanel()
TP_FILE_NAME = New System.Windows.Forms.TableLayoutPanel() TP_FILE_NAME = New System.Windows.Forms.TableLayoutPanel()
@@ -508,13 +509,13 @@ Namespace Editors
TP_FILE_NAME.Controls.Add(Me.OPT_FILE_NAME_ADD_DATE, 2, 0) TP_FILE_NAME.Controls.Add(Me.OPT_FILE_NAME_ADD_DATE, 2, 0)
TP_FILE_NAME.Controls.Add(Me.CH_FILE_NAME_CHANGE, 0, 0) TP_FILE_NAME.Controls.Add(Me.CH_FILE_NAME_CHANGE, 0, 0)
TP_FILE_NAME.Dock = System.Windows.Forms.DockStyle.Fill TP_FILE_NAME.Dock = System.Windows.Forms.DockStyle.Fill
TP_FILE_NAME.Location = New System.Drawing.Point(1, 53) TP_FILE_NAME.Location = New System.Drawing.Point(1, 79)
TP_FILE_NAME.Margin = New System.Windows.Forms.Padding(0) TP_FILE_NAME.Margin = New System.Windows.Forms.Padding(0)
TP_FILE_NAME.Name = "TP_FILE_NAME" TP_FILE_NAME.Name = "TP_FILE_NAME"
TP_FILE_NAME.RowCount = 1 TP_FILE_NAME.RowCount = 1
TP_FILE_NAME.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Percent, 100.0!)) TP_FILE_NAME.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Percent, 100.0!))
TP_FILE_NAME.Size = New System.Drawing.Size(574, 30) TP_FILE_NAME.Size = New System.Drawing.Size(574, 30)
TP_FILE_NAME.TabIndex = 2 TP_FILE_NAME.TabIndex = 3
' '
'OPT_FILE_NAME_REPLACE 'OPT_FILE_NAME_REPLACE
' '
@@ -566,14 +567,14 @@ Namespace Editors
TP_FILE_PATTERNS.Controls.Add(Me.OPT_FILE_DATE_START, 3, 0) TP_FILE_PATTERNS.Controls.Add(Me.OPT_FILE_DATE_START, 3, 0)
TP_FILE_PATTERNS.Controls.Add(Me.OPT_FILE_DATE_END, 4, 0) TP_FILE_PATTERNS.Controls.Add(Me.OPT_FILE_DATE_END, 4, 0)
TP_FILE_PATTERNS.Dock = System.Windows.Forms.DockStyle.Fill TP_FILE_PATTERNS.Dock = System.Windows.Forms.DockStyle.Fill
TP_FILE_PATTERNS.Location = New System.Drawing.Point(1, 84) TP_FILE_PATTERNS.Location = New System.Drawing.Point(1, 110)
TP_FILE_PATTERNS.Margin = New System.Windows.Forms.Padding(0) TP_FILE_PATTERNS.Margin = New System.Windows.Forms.Padding(0)
TP_FILE_PATTERNS.Name = "TP_FILE_PATTERNS" TP_FILE_PATTERNS.Name = "TP_FILE_PATTERNS"
TP_FILE_PATTERNS.RowCount = 1 TP_FILE_PATTERNS.RowCount = 1
TP_FILE_PATTERNS.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Percent, 100.0!)) TP_FILE_PATTERNS.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Percent, 100.0!))
TP_FILE_PATTERNS.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 29.0!)) TP_FILE_PATTERNS.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 29.0!))
TP_FILE_PATTERNS.Size = New System.Drawing.Size(574, 30) TP_FILE_PATTERNS.Size = New System.Drawing.Size(574, 30)
TP_FILE_PATTERNS.TabIndex = 3 TP_FILE_PATTERNS.TabIndex = 4
' '
'CH_FILE_DATE 'CH_FILE_DATE
' '
@@ -880,10 +881,10 @@ Namespace Editors
' '
Me.CH_DOWN_REPARSE_MISSING.AutoSize = True Me.CH_DOWN_REPARSE_MISSING.AutoSize = True
Me.CH_DOWN_REPARSE_MISSING.Dock = System.Windows.Forms.DockStyle.Fill Me.CH_DOWN_REPARSE_MISSING.Dock = System.Windows.Forms.DockStyle.Fill
Me.CH_DOWN_REPARSE_MISSING.Location = New System.Drawing.Point(4, 202) Me.CH_DOWN_REPARSE_MISSING.Location = New System.Drawing.Point(4, 228)
Me.CH_DOWN_REPARSE_MISSING.Name = "CH_DOWN_REPARSE_MISSING" Me.CH_DOWN_REPARSE_MISSING.Name = "CH_DOWN_REPARSE_MISSING"
Me.CH_DOWN_REPARSE_MISSING.Size = New System.Drawing.Size(568, 19) Me.CH_DOWN_REPARSE_MISSING.Size = New System.Drawing.Size(568, 19)
Me.CH_DOWN_REPARSE_MISSING.TabIndex = 7 Me.CH_DOWN_REPARSE_MISSING.TabIndex = 8
Me.CH_DOWN_REPARSE_MISSING.Text = "Trying to download missing posts using regular download" Me.CH_DOWN_REPARSE_MISSING.Text = "Trying to download missing posts using regular download"
TT_MAIN.SetToolTip(Me.CH_DOWN_REPARSE_MISSING, "If missing posts exist, the missing posts will attempt to be downloaded via user " & TT_MAIN.SetToolTip(Me.CH_DOWN_REPARSE_MISSING, "If missing posts exist, the missing posts will attempt to be downloaded via user " &
"download") "download")
@@ -925,6 +926,34 @@ Namespace Editors
TT_MAIN.SetToolTip(Me.CH_STD_EVERY, "Show notifications when download in standalone downloader is complete") TT_MAIN.SetToolTip(Me.CH_STD_EVERY, "Show notifications when download in standalone downloader is complete")
Me.CH_STD_EVERY.UseVisualStyleBackColor = True Me.CH_STD_EVERY.UseVisualStyleBackColor = True
' '
'CH_STD_YT_LOAD
'
Me.CH_STD_YT_LOAD.AutoSize = True
Me.CH_STD_YT_LOAD.Dock = System.Windows.Forms.DockStyle.Fill
Me.CH_STD_YT_LOAD.Location = New System.Drawing.Point(4, 166)
Me.CH_STD_YT_LOAD.Name = "CH_STD_YT_LOAD"
Me.CH_STD_YT_LOAD.Padding = New System.Windows.Forms.Padding(100, 0, 0, 0)
Me.CH_STD_YT_LOAD.Size = New System.Drawing.Size(568, 19)
Me.CH_STD_YT_LOAD.TabIndex = 6
Me.CH_STD_YT_LOAD.Text = "Load downloaded YouTube videos to the form"
TT_MAIN.SetToolTip(Me.CH_STD_YT_LOAD, "If checked, downloaded YouTube videos will be loaded to the form. Otherwise, all " &
"downloaded data will be loaded to the form except YouTube data.")
Me.CH_STD_YT_LOAD.UseVisualStyleBackColor = True
'
'CH_STD_YT_REMOVE
'
Me.CH_STD_YT_REMOVE.AutoSize = True
Me.CH_STD_YT_REMOVE.Dock = System.Windows.Forms.DockStyle.Fill
Me.CH_STD_YT_REMOVE.Location = New System.Drawing.Point(4, 192)
Me.CH_STD_YT_REMOVE.Name = "CH_STD_YT_REMOVE"
Me.CH_STD_YT_REMOVE.Padding = New System.Windows.Forms.Padding(100, 0, 0, 0)
Me.CH_STD_YT_REMOVE.Size = New System.Drawing.Size(568, 19)
Me.CH_STD_YT_REMOVE.TabIndex = 7
Me.CH_STD_YT_REMOVE.Text = "Clear YouTube videos when clearing the list"
TT_MAIN.SetToolTip(Me.CH_STD_YT_REMOVE, "If checked, YouTube videos will also be removed from the list. This action will a" &
"lso affect the standalone 'YouTubeDownloader' app.")
Me.CH_STD_YT_REMOVE.UseVisualStyleBackColor = True
'
'TP_CHANNELS_IMGS 'TP_CHANNELS_IMGS
' '
TP_CHANNELS_IMGS.ColumnCount = 2 TP_CHANNELS_IMGS.ColumnCount = 2
@@ -1265,18 +1294,20 @@ Namespace Editors
TP_DOWNLOADING.CellBorderStyle = System.Windows.Forms.TableLayoutPanelCellBorderStyle.[Single] TP_DOWNLOADING.CellBorderStyle = System.Windows.Forms.TableLayoutPanelCellBorderStyle.[Single]
TP_DOWNLOADING.ColumnCount = 1 TP_DOWNLOADING.ColumnCount = 1
TP_DOWNLOADING.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Percent, 100.0!)) TP_DOWNLOADING.ColumnStyles.Add(New System.Windows.Forms.ColumnStyle(System.Windows.Forms.SizeType.Percent, 100.0!))
TP_DOWNLOADING.Controls.Add(TP_FILE_NAME, 0, 2) TP_DOWNLOADING.Controls.Add(TP_FILE_NAME, 0, 3)
TP_DOWNLOADING.Controls.Add(TP_FILE_PATTERNS, 0, 3) TP_DOWNLOADING.Controls.Add(TP_FILE_PATTERNS, 0, 4)
TP_DOWNLOADING.Controls.Add(Me.TXT_SCRIPT, 0, 4) TP_DOWNLOADING.Controls.Add(Me.TXT_SCRIPT, 0, 5)
TP_DOWNLOADING.Controls.Add(Me.CH_UDESCR_UP, 0, 0) TP_DOWNLOADING.Controls.Add(Me.CH_UDESCR_UP, 0, 0)
TP_DOWNLOADING.Controls.Add(Me.TXT_DOWN_COMPLETE_SCRIPT, 0, 5) TP_DOWNLOADING.Controls.Add(Me.TXT_DOWN_COMPLETE_SCRIPT, 0, 6)
TP_DOWNLOADING.Controls.Add(TP_MISSING_DATA, 0, 6) TP_DOWNLOADING.Controls.Add(TP_MISSING_DATA, 0, 7)
TP_DOWNLOADING.Controls.Add(Me.CH_DOWN_REPARSE_MISSING, 0, 7) TP_DOWNLOADING.Controls.Add(Me.CH_DOWN_REPARSE_MISSING, 0, 8)
TP_DOWNLOADING.Controls.Add(Me.CH_UNAME_UP, 0, 1) TP_DOWNLOADING.Controls.Add(Me.CH_UNAME_UP, 0, 1)
TP_DOWNLOADING.Controls.Add(Me.CH_UICON_UP, 0, 2)
TP_DOWNLOADING.Dock = System.Windows.Forms.DockStyle.Fill TP_DOWNLOADING.Dock = System.Windows.Forms.DockStyle.Fill
TP_DOWNLOADING.Location = New System.Drawing.Point(0, 0) TP_DOWNLOADING.Location = New System.Drawing.Point(0, 0)
TP_DOWNLOADING.Name = "TP_DOWNLOADING" TP_DOWNLOADING.Name = "TP_DOWNLOADING"
TP_DOWNLOADING.RowCount = 9 TP_DOWNLOADING.RowCount = 10
TP_DOWNLOADING.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!))
TP_DOWNLOADING.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!)) TP_DOWNLOADING.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!))
TP_DOWNLOADING.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!)) TP_DOWNLOADING.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 25.0!))
TP_DOWNLOADING.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 30.0!)) TP_DOWNLOADING.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 30.0!))
@@ -1305,12 +1336,12 @@ Namespace Editors
Me.TXT_SCRIPT.CaptionWidth = 120.0R Me.TXT_SCRIPT.CaptionWidth = 120.0R
Me.TXT_SCRIPT.ChangeControlsEnableOnCheckedChange = False Me.TXT_SCRIPT.ChangeControlsEnableOnCheckedChange = False
Me.TXT_SCRIPT.Dock = System.Windows.Forms.DockStyle.Fill Me.TXT_SCRIPT.Dock = System.Windows.Forms.DockStyle.Fill
Me.TXT_SCRIPT.Location = New System.Drawing.Point(4, 118) Me.TXT_SCRIPT.Location = New System.Drawing.Point(4, 144)
Me.TXT_SCRIPT.Name = "TXT_SCRIPT" Me.TXT_SCRIPT.Name = "TXT_SCRIPT"
Me.TXT_SCRIPT.PlaceholderEnabled = True Me.TXT_SCRIPT.PlaceholderEnabled = True
Me.TXT_SCRIPT.PlaceholderText = "Enter script path here..." Me.TXT_SCRIPT.PlaceholderText = "Enter script path here..."
Me.TXT_SCRIPT.Size = New System.Drawing.Size(568, 22) Me.TXT_SCRIPT.Size = New System.Drawing.Size(568, 22)
Me.TXT_SCRIPT.TabIndex = 4 Me.TXT_SCRIPT.TabIndex = 5
' '
'TXT_DOWN_COMPLETE_SCRIPT 'TXT_DOWN_COMPLETE_SCRIPT
' '
@@ -1320,12 +1351,12 @@ Namespace Editors
Me.TXT_DOWN_COMPLETE_SCRIPT.CaptionToolTipText = "This command will be executed after all downloads are completed" Me.TXT_DOWN_COMPLETE_SCRIPT.CaptionToolTipText = "This command will be executed after all downloads are completed"
Me.TXT_DOWN_COMPLETE_SCRIPT.CaptionWidth = 120.0R Me.TXT_DOWN_COMPLETE_SCRIPT.CaptionWidth = 120.0R
Me.TXT_DOWN_COMPLETE_SCRIPT.Dock = System.Windows.Forms.DockStyle.Fill Me.TXT_DOWN_COMPLETE_SCRIPT.Dock = System.Windows.Forms.DockStyle.Fill
Me.TXT_DOWN_COMPLETE_SCRIPT.Location = New System.Drawing.Point(4, 147) Me.TXT_DOWN_COMPLETE_SCRIPT.Location = New System.Drawing.Point(4, 173)
Me.TXT_DOWN_COMPLETE_SCRIPT.Name = "TXT_DOWN_COMPLETE_SCRIPT" Me.TXT_DOWN_COMPLETE_SCRIPT.Name = "TXT_DOWN_COMPLETE_SCRIPT"
Me.TXT_DOWN_COMPLETE_SCRIPT.PlaceholderEnabled = True Me.TXT_DOWN_COMPLETE_SCRIPT.PlaceholderEnabled = True
Me.TXT_DOWN_COMPLETE_SCRIPT.PlaceholderText = "Enter command here..." Me.TXT_DOWN_COMPLETE_SCRIPT.PlaceholderText = "Enter command here..."
Me.TXT_DOWN_COMPLETE_SCRIPT.Size = New System.Drawing.Size(568, 22) Me.TXT_DOWN_COMPLETE_SCRIPT.Size = New System.Drawing.Size(568, 22)
Me.TXT_DOWN_COMPLETE_SCRIPT.TabIndex = 5 Me.TXT_DOWN_COMPLETE_SCRIPT.TabIndex = 6
' '
'TP_MISSING_DATA 'TP_MISSING_DATA
' '
@@ -1336,14 +1367,14 @@ Namespace Editors
TP_MISSING_DATA.Controls.Add(Me.CH_ADD_MISSING_TO_LOG, 0, 0) TP_MISSING_DATA.Controls.Add(Me.CH_ADD_MISSING_TO_LOG, 0, 0)
TP_MISSING_DATA.Controls.Add(Me.CH_ADD_MISSING_ERROS_TO_LOG, 1, 0) TP_MISSING_DATA.Controls.Add(Me.CH_ADD_MISSING_ERROS_TO_LOG, 1, 0)
TP_MISSING_DATA.Dock = System.Windows.Forms.DockStyle.Fill TP_MISSING_DATA.Dock = System.Windows.Forms.DockStyle.Fill
TP_MISSING_DATA.Location = New System.Drawing.Point(1, 173) TP_MISSING_DATA.Location = New System.Drawing.Point(1, 199)
TP_MISSING_DATA.Margin = New System.Windows.Forms.Padding(0) TP_MISSING_DATA.Margin = New System.Windows.Forms.Padding(0)
TP_MISSING_DATA.Name = "TP_MISSING_DATA" TP_MISSING_DATA.Name = "TP_MISSING_DATA"
TP_MISSING_DATA.RowCount = 1 TP_MISSING_DATA.RowCount = 1
TP_MISSING_DATA.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Percent, 100.0!)) TP_MISSING_DATA.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Percent, 100.0!))
TP_MISSING_DATA.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 24.0!)) TP_MISSING_DATA.RowStyles.Add(New System.Windows.Forms.RowStyle(System.Windows.Forms.SizeType.Absolute, 24.0!))
TP_MISSING_DATA.Size = New System.Drawing.Size(574, 25) TP_MISSING_DATA.Size = New System.Drawing.Size(574, 25)
TP_MISSING_DATA.TabIndex = 6 TP_MISSING_DATA.TabIndex = 7
' '
'CH_UNAME_UP 'CH_UNAME_UP
' '
@@ -1352,7 +1383,7 @@ Namespace Editors
Me.CH_UNAME_UP.Location = New System.Drawing.Point(4, 30) Me.CH_UNAME_UP.Location = New System.Drawing.Point(4, 30)
Me.CH_UNAME_UP.Name = "CH_UNAME_UP" Me.CH_UNAME_UP.Name = "CH_UNAME_UP"
Me.CH_UNAME_UP.Size = New System.Drawing.Size(568, 19) Me.CH_UNAME_UP.Size = New System.Drawing.Size(568, 19)
Me.CH_UNAME_UP.TabIndex = 7 Me.CH_UNAME_UP.TabIndex = 1
Me.CH_UNAME_UP.Text = "Update user site name every time" Me.CH_UNAME_UP.Text = "Update user site name every time"
Me.CH_UNAME_UP.UseVisualStyleBackColor = True Me.CH_UNAME_UP.UseVisualStyleBackColor = True
' '
@@ -1843,33 +1874,16 @@ Namespace Editors
Me.CONTAINER_MAIN.TabIndex = 0 Me.CONTAINER_MAIN.TabIndex = 0
Me.CONTAINER_MAIN.TopToolStripPanelVisible = False Me.CONTAINER_MAIN.TopToolStripPanelVisible = False
' '
'CH_STD_YT_LOAD 'CH_UICON_UP
' '
Me.CH_STD_YT_LOAD.AutoSize = True Me.CH_UICON_UP.AutoSize = True
Me.CH_STD_YT_LOAD.Dock = System.Windows.Forms.DockStyle.Fill Me.CH_UICON_UP.Dock = System.Windows.Forms.DockStyle.Fill
Me.CH_STD_YT_LOAD.Location = New System.Drawing.Point(4, 166) Me.CH_UICON_UP.Location = New System.Drawing.Point(4, 56)
Me.CH_STD_YT_LOAD.Name = "CH_STD_YT_LOAD" Me.CH_UICON_UP.Name = "CH_UICON_UP"
Me.CH_STD_YT_LOAD.Padding = New System.Windows.Forms.Padding(100, 0, 0, 0) Me.CH_UICON_UP.Size = New System.Drawing.Size(568, 19)
Me.CH_STD_YT_LOAD.Size = New System.Drawing.Size(568, 19) Me.CH_UICON_UP.TabIndex = 2
Me.CH_STD_YT_LOAD.TabIndex = 6 Me.CH_UICON_UP.Text = "Update user icon and banner every time (where supported)"
Me.CH_STD_YT_LOAD.Text = "Load downloaded YouTube videos to the form" Me.CH_UICON_UP.UseVisualStyleBackColor = True
TT_MAIN.SetToolTip(Me.CH_STD_YT_LOAD, "If checked, downloaded YouTube videos will be loaded to the form. Otherwise, all " &
"downloaded data will be loaded to the form except YouTube data.")
Me.CH_STD_YT_LOAD.UseVisualStyleBackColor = True
'
'CH_STD_YT_REMOVE
'
Me.CH_STD_YT_REMOVE.AutoSize = True
Me.CH_STD_YT_REMOVE.Dock = System.Windows.Forms.DockStyle.Fill
Me.CH_STD_YT_REMOVE.Location = New System.Drawing.Point(4, 192)
Me.CH_STD_YT_REMOVE.Name = "CH_STD_YT_REMOVE"
Me.CH_STD_YT_REMOVE.Padding = New System.Windows.Forms.Padding(100, 0, 0, 0)
Me.CH_STD_YT_REMOVE.Size = New System.Drawing.Size(568, 19)
Me.CH_STD_YT_REMOVE.TabIndex = 7
Me.CH_STD_YT_REMOVE.Text = "Clear YouTube videos when clearing the list"
TT_MAIN.SetToolTip(Me.CH_STD_YT_REMOVE, "If checked, YouTube videos will also be removed from the list. This action will a" &
"lso affect the standalone 'YouTubeDownloader' app.")
Me.CH_STD_YT_REMOVE.UseVisualStyleBackColor = True
' '
'GlobalSettingsForm 'GlobalSettingsForm
' '
@@ -2041,5 +2055,6 @@ Namespace Editors
Private WithEvents CH_STD_UPDATE_YT_PATH As CheckBox Private WithEvents CH_STD_UPDATE_YT_PATH As CheckBox
Private WithEvents CH_STD_YT_LOAD As CheckBox Private WithEvents CH_STD_YT_LOAD As CheckBox
Private WithEvents CH_STD_YT_REMOVE As CheckBox Private WithEvents CH_STD_YT_REMOVE As CheckBox
Private WithEvents CH_UICON_UP As CheckBox
End Class End Class
End Namespace End Namespace

2
SCrawler/Editors/GlobalSettingsForm.vb
View File

@@ -89,6 +89,7 @@ Namespace Editors
'Downloading 'Downloading
CH_UDESCR_UP.Checked = .UpdateUserDescriptionEveryTime CH_UDESCR_UP.Checked = .UpdateUserDescriptionEveryTime
CH_UNAME_UP.Checked = .UserSiteNameUpdateEveryTime CH_UNAME_UP.Checked = .UserSiteNameUpdateEveryTime
CH_UICON_UP.Checked = .UpdateUserIconBannerEveryTime
TXT_SCRIPT.Checked = .ScriptData.Attribute TXT_SCRIPT.Checked = .ScriptData.Attribute
TXT_SCRIPT.Text = .ScriptData.Value TXT_SCRIPT.Text = .ScriptData.Value
TXT_DOWN_COMPLETE_SCRIPT.Text = .DownloadsCompleteCommand TXT_DOWN_COMPLETE_SCRIPT.Text = .DownloadsCompleteCommand
@@ -244,6 +245,7 @@ Namespace Editors
'Downloading 'Downloading
.UpdateUserDescriptionEveryTime.Value = CH_UDESCR_UP.Checked .UpdateUserDescriptionEveryTime.Value = CH_UDESCR_UP.Checked
.UserSiteNameUpdateEveryTime.Value = CH_UNAME_UP.Checked .UserSiteNameUpdateEveryTime.Value = CH_UNAME_UP.Checked
.UpdateUserIconBannerEveryTime.Value = CH_UICON_UP.Checked
.ScriptData.Value = TXT_SCRIPT.Text .ScriptData.Value = TXT_SCRIPT.Text
.ScriptData.Attribute.Value = TXT_SCRIPT.Checked .ScriptData.Attribute.Value = TXT_SCRIPT.Checked
.DownloadsCompleteCommand.Value = TXT_DOWN_COMPLETE_SCRIPT.Text .DownloadsCompleteCommand.Value = TXT_DOWN_COMPLETE_SCRIPT.Text

14
SCrawler/Editors/UsersInfoForm.vb
View File

@@ -492,13 +492,23 @@ Namespace Editors
If Not info.IsEmptyString Then MsgBoxE({info, "User information"}) If Not info.IsEmptyString Then MsgBoxE({info, "User information"})
End Sub End Sub
Private Sub CONTEXT_BTT_OPEN_FOLDER_Click(sender As Object, e As EventArgs) Handles CONTEXT_BTT_OPEN_FOLDER.Click Private Sub CONTEXT_BTT_OPEN_FOLDER_Click(sender As Object, e As EventArgs) Handles CONTEXT_BTT_OPEN_FOLDER.Click
Dim u As UserOpt = GetUserFromList() OpenUserFolder()
If Not u Is Nothing Then u.User.OpenFolder()
End Sub End Sub
Private Sub CONTEXT_BTT_OPEN_SITE_Click(sender As Object, e As EventArgs) Handles CONTEXT_BTT_OPEN_SITE.Click Private Sub CONTEXT_BTT_OPEN_SITE_Click(sender As Object, e As EventArgs) Handles CONTEXT_BTT_OPEN_SITE.Click
Dim u As UserOpt = GetUserFromList() Dim u As UserOpt = GetUserFromList()
If Not u Is Nothing Then u.User.OpenSite() If Not u Is Nothing Then u.User.OpenSite()
End Sub End Sub
#End Region
#Region "List handlers"
Private Sub LIST_DATA_MouseDoubleClick(sender As Object, e As MouseEventArgs) Handles LIST_DATA.MouseDoubleClick
OpenUserFolder()
End Sub
#End Region
#Region "Functions"
Private Sub OpenUserFolder()
Dim u As UserOpt = GetUserFromList()
If Not u Is Nothing Then u.User.OpenFolder()
End Sub
#End Region #End Region
End Class End Class
End Namespace End Namespace

2
SCrawler/ListImagesLoader.vb
View File

@@ -122,6 +122,8 @@ Friend Class ListImagesLoader
UserDataList.Clear() UserDataList.Clear()
UpdateInProgress = False UpdateInProgress = False
End If End If
Else
UpdateInProgress = False
End If End If
Else Else
Dim t As New List(Of Task) Dim t As New List(Of Task)

13
SCrawler/MainFrame.Designer.vb generated
View File

@@ -125,6 +125,7 @@ Partial Public Class MainFrame : Inherits System.Windows.Forms.Form
Me.BTT_TRAY_SHOW_HIDE = New System.Windows.Forms.ToolStripMenuItem() Me.BTT_TRAY_SHOW_HIDE = New System.Windows.Forms.ToolStripMenuItem()
Me.BTT_TRAY_CLOSE = New System.Windows.Forms.ToolStripMenuItem() Me.BTT_TRAY_CLOSE = New System.Windows.Forms.ToolStripMenuItem()
Me.BTT_TRAY_CLOSE_NO_SCRIPT = New System.Windows.Forms.ToolStripMenuItem() Me.BTT_TRAY_CLOSE_NO_SCRIPT = New System.Windows.Forms.ToolStripMenuItem()
Me.BTT_TRAY_DOWNLOADER = New System.Windows.Forms.ToolStripMenuItem()
SEP_1 = New System.Windows.Forms.ToolStripSeparator() SEP_1 = New System.Windows.Forms.ToolStripSeparator()
SEP_2 = New System.Windows.Forms.ToolStripSeparator() SEP_2 = New System.Windows.Forms.ToolStripSeparator()
CONTEXT_SEP_1 = New System.Windows.Forms.ToolStripSeparator() CONTEXT_SEP_1 = New System.Windows.Forms.ToolStripSeparator()
@@ -826,9 +827,9 @@ Partial Public Class MainFrame : Inherits System.Windows.Forms.Form
' '
'TRAY_CONTEXT 'TRAY_CONTEXT
' '
Me.TRAY_CONTEXT.Items.AddRange(New System.Windows.Forms.ToolStripItem() {Me.BTT_TRAY_PAUSE_AUTOMATION, Me.BTT_TRAY_SILENT_MODE, Me.BTT_TRAY_FEED_SHOW, Me.BTT_TRAY_CHANNELS, TRAY_SEP_1, Me.BTT_TRAY_SHOW_HIDE, TRAY_SEP_2, Me.BTT_TRAY_CLOSE, Me.BTT_TRAY_CLOSE_NO_SCRIPT}) Me.TRAY_CONTEXT.Items.AddRange(New System.Windows.Forms.ToolStripItem() {Me.BTT_TRAY_PAUSE_AUTOMATION, Me.BTT_TRAY_SILENT_MODE, Me.BTT_TRAY_FEED_SHOW, Me.BTT_TRAY_CHANNELS, Me.BTT_TRAY_DOWNLOADER, TRAY_SEP_1, Me.BTT_TRAY_SHOW_HIDE, TRAY_SEP_2, Me.BTT_TRAY_CLOSE, Me.BTT_TRAY_CLOSE_NO_SCRIPT})
Me.TRAY_CONTEXT.Name = "TRAY_CONTEXT" Me.TRAY_CONTEXT.Name = "TRAY_CONTEXT"
Me.TRAY_CONTEXT.Size = New System.Drawing.Size(171, 170) Me.TRAY_CONTEXT.Size = New System.Drawing.Size(171, 192)
' '
'BTT_TRAY_PAUSE_AUTOMATION 'BTT_TRAY_PAUSE_AUTOMATION
' '
@@ -892,6 +893,13 @@ Partial Public Class MainFrame : Inherits System.Windows.Forms.Form
Me.BTT_TRAY_CLOSE_NO_SCRIPT.ToolTipText = "Close the program without executing the script" Me.BTT_TRAY_CLOSE_NO_SCRIPT.ToolTipText = "Close the program without executing the script"
Me.BTT_TRAY_CLOSE_NO_SCRIPT.Visible = False Me.BTT_TRAY_CLOSE_NO_SCRIPT.Visible = False
' '
'BTT_TRAY_DOWNLOADER
'
Me.BTT_TRAY_DOWNLOADER.Image = Global.SCrawler.My.Resources.Resources.ArrowDownPic_Blue_24
Me.BTT_TRAY_DOWNLOADER.Name = "BTT_TRAY_DOWNLOADER"
Me.BTT_TRAY_DOWNLOADER.Size = New System.Drawing.Size(170, 22)
Me.BTT_TRAY_DOWNLOADER.Text = "Downloader"
'
'MainFrame 'MainFrame
' '
Me.AutoScaleDimensions = New System.Drawing.SizeF(6.0!, 13.0!) Me.AutoScaleDimensions = New System.Drawing.SizeF(6.0!, 13.0!)
@@ -996,4 +1004,5 @@ Partial Public Class MainFrame : Inherits System.Windows.Forms.Form
Private WithEvents BTT_TRAY_FEED_SHOW As ToolStripMenuItem Private WithEvents BTT_TRAY_FEED_SHOW As ToolStripMenuItem
Friend WithEvents MENU_DOWN_ALL As ToolStripDropDownButton Friend WithEvents MENU_DOWN_ALL As ToolStripDropDownButton
Private WithEvents BTT_TRAY_CHANNELS As ToolStripMenuItem Private WithEvents BTT_TRAY_CHANNELS As ToolStripMenuItem
Private WithEvents BTT_TRAY_DOWNLOADER As ToolStripMenuItem
End Class End Class

8
SCrawler/MainFrame.vb
View File

@@ -452,9 +452,9 @@ CloseResume:
Downloader.AddRange(Settings.GetUsers(UserExistsPredicate), e.IncludeInTheFeed) Downloader.AddRange(Settings.GetUsers(UserExistsPredicate), e.IncludeInTheFeed)
End Sub End Sub
Private Sub BTT_DOWN_SITE_FULL_KeyClick(sender As Object, e As MyKeyEventArgs) Handles BTT_DOWN_SITE_FULL.KeyClick Private Sub BTT_DOWN_SITE_FULL_KeyClick(sender As Object, e As MyKeyEventArgs) Handles BTT_DOWN_SITE_FULL.KeyClick
DownloadSiteFull(False, e.IncludeInTheFeed) DownloadSiteFull(False, e.IncludeInTheFeed, e.Shift)
End Sub End Sub
Private Sub DownloadSiteFull(ByVal ReadyForDownloadOnly As Boolean, ByVal IncludeInTheFeed As Boolean) Private Sub DownloadSiteFull(ByVal ReadyForDownloadOnly As Boolean, ByVal IncludeInTheFeed As Boolean, Optional ByVal IgnoreExists As Boolean = False)
Using f As New SiteSelectionForm(Settings.LatestDownloadedSites.ValuesList) Using f As New SiteSelectionForm(Settings.LatestDownloadedSites.ValuesList)
f.ShowDialog() f.ShowDialog()
If f.DialogResult = DialogResult.OK Then If f.DialogResult = DialogResult.OK Then
@@ -462,7 +462,7 @@ CloseResume:
Settings.LatestDownloadedSites.AddRange(f.SelectedSites) Settings.LatestDownloadedSites.AddRange(f.SelectedSites)
Settings.LatestDownloadedSites.Update() Settings.LatestDownloadedSites.Update()
If f.SelectedSites.Count > 0 Then If f.SelectedSites.Count > 0 Then
Downloader.AddRange(Settings.GetUsers(Function(u) f.SelectedSites.Contains(u.Site) And u.Exists And Downloader.AddRange(Settings.GetUsers(Function(u) f.SelectedSites.Contains(u.Site) And (u.Exists Or IgnoreExists) And
(Not ReadyForDownloadOnly Or u.ReadyForDownload)), IncludeInTheFeed) (Not ReadyForDownloadOnly Or u.ReadyForDownload)), IncludeInTheFeed)
End If End If
End If End If
@@ -513,7 +513,7 @@ CloseResume:
TrayIcon.ContextMenuStrip.Hide() TrayIcon.ContextMenuStrip.Hide()
MainFrameObj.PauseButtons.UpdatePauseButtons() MainFrameObj.PauseButtons.UpdatePauseButtons()
End Sub End Sub
Private Sub BTT_DOWN_VIDEO_Click(sender As Object, e As EventArgs) Handles BTT_DOWN_VIDEO.Click Private Sub BTT_DOWN_VIDEO_Click(sender As Object, e As EventArgs) Handles BTT_DOWN_VIDEO.Click, BTT_TRAY_DOWNLOADER.Click
VideoDownloader.FormShow() VideoDownloader.FormShow()
End Sub End Sub
Private Sub BTT_DOWN_STOP_Click(sender As Object, e As EventArgs) Handles BTT_DOWN_STOP.Click Private Sub BTT_DOWN_STOP_Click(sender As Object, e As EventArgs) Handles BTT_DOWN_STOP.Click

4
SCrawler/My Project/AssemblyInfo.vb
View File

@@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices
' by using the '*' as shown below: ' by using the '*' as shown below:
' <Assembly: AssemblyVersion("1.0.*")> ' <Assembly: AssemblyVersion("1.0.*")>
<Assembly: AssemblyVersion("2023.5.12.0")> <Assembly: AssemblyVersion("2023.6.5.0")>
<Assembly: AssemblyFileVersion("2023.5.12.0")> <Assembly: AssemblyFileVersion("2023.6.5.0")>
<Assembly: NeutralResourcesLanguage("en")> <Assembly: NeutralResourcesLanguage("en")>

14
SCrawler/PluginsEnvironment/Hosts/DownloadableMediaHost.vb
View File

@@ -23,6 +23,15 @@ Namespace Plugin.Hosts
End Property End Property
Friend Property Instance As UserDataBase Friend Property Instance As UserDataBase
Friend ReadOnly Property ExternalSource As IDownloadableMedia = Nothing Friend ReadOnly Property ExternalSource As IDownloadableMedia = Nothing
Public Overrides ReadOnly Property Exists As Boolean
Get
If SiteKey = API.YouTube.YouTubeSiteKey Then
Return MyBase.Exists
Else
Return _Exists
End If
End Get
End Property
Public Overrides Property File As SFile Public Overrides Property File As SFile
Get Get
Return _File Return _File
@@ -128,7 +137,7 @@ Namespace Plugin.Hosts
End Sub End Sub
Public Overrides Sub Load(ByVal f As SFile) Public Overrides Sub Load(ByVal f As SFile)
MyBase.Load(f) MyBase.Load(f)
If _Exists Then _Exists = File.Exists If _Exists Then _Exists = Not MediaState = UserMediaStates.Downloaded OrElse File.Exists
End Sub End Sub
Public Overrides Sub Save() Public Overrides Sub Save()
If FileSettings.IsEmptyString Then If FileSettings.IsEmptyString Then
@@ -142,6 +151,9 @@ Namespace Plugin.Hosts
x.Save(FileSettings) x.Save(FileSettings)
End Using End Using
End Sub End Sub
Public Overrides Function GetHashCode() As Integer
Return URL.GetHashCode
End Function
Protected Overrides Sub Dispose(ByVal disposing As Boolean) Protected Overrides Sub Dispose(ByVal disposing As Boolean)
If Not disposedValue And disposing Then Instance.DisposeIfReady() : ExternalSource.DisposeIfReady(False) If Not disposedValue And disposing Then Instance.DisposeIfReady() : ExternalSource.DisposeIfReady(False)
MyBase.Dispose(disposing) MyBase.Dispose(disposing)

2
SCrawler/SettingsCLS.vb
View File

@@ -238,6 +238,7 @@ Friend Class SettingsCLS : Implements IDownloaderSettings, IDisposable
FromChannelDownloadTopUse = New XMLValue(Of Boolean)("FromChannelDownloadTopUse", False, MyXML, n) FromChannelDownloadTopUse = New XMLValue(Of Boolean)("FromChannelDownloadTopUse", False, MyXML, n)
FromChannelCopyImageToUser = New XMLValue(Of Boolean)("FromChannelCopyImageToUser", True, MyXML, n) FromChannelCopyImageToUser = New XMLValue(Of Boolean)("FromChannelCopyImageToUser", True, MyXML, n)
UpdateUserDescriptionEveryTime = New XMLValue(Of Boolean)("UpdateUserDescriptionEveryTime", True, MyXML, n) UpdateUserDescriptionEveryTime = New XMLValue(Of Boolean)("UpdateUserDescriptionEveryTime", True, MyXML, n)
UpdateUserIconBannerEveryTime = New XMLValue(Of Boolean)("UpdateUserIconBannerEveryTime", True, MyXML, n)
ScriptData = New XMLValueAttribute(Of String, Boolean)("ScriptData", "Use",,, MyXML, n) ScriptData = New XMLValueAttribute(Of String, Boolean)("ScriptData", "Use",,, MyXML, n)
n = {"Users", "FileName"} n = {"Users", "FileName"}
@@ -720,6 +721,7 @@ Friend Class SettingsCLS : Implements IDownloaderSettings, IDisposable
Friend ReadOnly Property FromChannelDownloadTopUse As XMLValue(Of Boolean) Friend ReadOnly Property FromChannelDownloadTopUse As XMLValue(Of Boolean)
Friend ReadOnly Property FromChannelCopyImageToUser As XMLValue(Of Boolean) Friend ReadOnly Property FromChannelCopyImageToUser As XMLValue(Of Boolean)
Friend ReadOnly Property UpdateUserDescriptionEveryTime As XMLValue(Of Boolean) Friend ReadOnly Property UpdateUserDescriptionEveryTime As XMLValue(Of Boolean)
Friend ReadOnly Property UpdateUserIconBannerEveryTime As XMLValue(Of Boolean)
#Region "File naming" #Region "File naming"
Friend ReadOnly Property FileAddDateToFileName As XMLValue(Of Boolean) Friend ReadOnly Property FileAddDateToFileName As XMLValue(Of Boolean)
Friend ReadOnly Property FileAddTimeToFileName As XMLValue(Of Boolean) Friend ReadOnly Property FileAddTimeToFileName As XMLValue(Of Boolean)