Sorting, multi category, multi resolution
After a file has been downloaded a callback function can now be called.
The callback function I call checks to see if the resolution of the image appears in the collection of resolutions that has been entered in the configuration file and deletes/moves accordingly.
If a file can not be read (which I have noticed happens sometimes), it is removed, not copied and not archived so that it can be retried later.
4grab got a new command-line option, -s --sorter, to sort out old images, running python sorter.py has the same effect, but this seemed pretties.
theoretically multiple categories could now be entered into the configuration file seperated by ',', but this hasn't been tested yet.
mutliple resolutions could be entered into the configuration file, seperated by ',' like so: 1680x1050,1920x1200.
Configuration now checks to see if all the necessary properties are available in the configuration file, if one is missing, it tries to create it.
2010-03-19 00:18:04 +01:00
|
|
|
######################################################################
|
|
|
|
# Copyright 2009, 2010 ryuslash
|
|
|
|
#
|
|
|
|
# This file is part of 4grab.
|
|
|
|
#
|
|
|
|
# 4grab is free software: you can redistribute it and/or modify
|
|
|
|
# it under the terms of the GNU General Public License as published by
|
|
|
|
# the Free Software Foundation, either version 3 of the License, or
|
|
|
|
# (at your option) any later version.
|
|
|
|
#
|
|
|
|
# 4grab is distributed in the hope that it will be useful,
|
|
|
|
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
|
|
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
|
|
# GNU General Public License for more details.
|
|
|
|
#
|
|
|
|
# You should have received a copy of the GNU General Public License
|
|
|
|
# along with 4grab. If not, see <http://www.gnu.org/licenses/>.
|
|
|
|
######################################################################
|
|
|
|
|
|
|
|
import config
|
|
|
|
import Image
|
|
|
|
import shutil
|
|
|
|
import os
|
2010-03-22 15:17:55 +01:00
|
|
|
import datetime
|
2010-04-19 09:16:04 +02:00
|
|
|
import backend
|
Sorting, multi category, multi resolution
After a file has been downloaded a callback function can now be called.
The callback function I call checks to see if the resolution of the image appears in the collection of resolutions that has been entered in the configuration file and deletes/moves accordingly.
If a file can not be read (which I have noticed happens sometimes), it is removed, not copied and not archived so that it can be retried later.
4grab got a new command-line option, -s --sorter, to sort out old images, running python sorter.py has the same effect, but this seemed pretties.
theoretically multiple categories could now be entered into the configuration file seperated by ',', but this hasn't been tested yet.
mutliple resolutions could be entered into the configuration file, seperated by ',' like so: 1680x1050,1920x1200.
Configuration now checks to see if all the necessary properties are available in the configuration file, if one is missing, it tries to create it.
2010-03-19 00:18:04 +01:00
|
|
|
|
2010-04-22 22:24:38 +02:00
|
|
|
import util
|
|
|
|
|
Sorting, multi category, multi resolution
After a file has been downloaded a callback function can now be called.
The callback function I call checks to see if the resolution of the image appears in the collection of resolutions that has been entered in the configuration file and deletes/moves accordingly.
If a file can not be read (which I have noticed happens sometimes), it is removed, not copied and not archived so that it can be retried later.
4grab got a new command-line option, -s --sorter, to sort out old images, running python sorter.py has the same effect, but this seemed pretties.
theoretically multiple categories could now be entered into the configuration file seperated by ',', but this hasn't been tested yet.
mutliple resolutions could be entered into the configuration file, seperated by ',' like so: 1680x1050,1920x1200.
Configuration now checks to see if all the necessary properties are available in the configuration file, if one is missing, it tries to create it.
2010-03-19 00:18:04 +01:00
|
|
|
def dummy_option_creator(value1, value2): pass
|
|
|
|
config._optioncreator = dummy_option_creator
|
|
|
|
|
|
|
|
class Sorter:
|
|
|
|
|
|
|
|
def __init__(self):
|
|
|
|
self.conf = config.Configuration()
|
|
|
|
self.resolutions = self.conf.get_resolutions()
|
|
|
|
|
|
|
|
def act(self, filename):
|
2010-04-22 22:24:38 +02:00
|
|
|
util.log(util.LogType.Msg, "sorter is acting", filename)
|
|
|
|
|
Sorting, multi category, multi resolution
After a file has been downloaded a callback function can now be called.
The callback function I call checks to see if the resolution of the image appears in the collection of resolutions that has been entered in the configuration file and deletes/moves accordingly.
If a file can not be read (which I have noticed happens sometimes), it is removed, not copied and not archived so that it can be retried later.
4grab got a new command-line option, -s --sorter, to sort out old images, running python sorter.py has the same effect, but this seemed pretties.
theoretically multiple categories could now be entered into the configuration file seperated by ',', but this hasn't been tested yet.
mutliple resolutions could be entered into the configuration file, seperated by ',' like so: 1680x1050,1920x1200.
Configuration now checks to see if all the necessary properties are available in the configuration file, if one is missing, it tries to create it.
2010-03-19 00:18:04 +01:00
|
|
|
download_base = self.conf.get_download_location()
|
2010-03-19 16:08:39 +01:00
|
|
|
retval = True
|
Sorting, multi category, multi resolution
After a file has been downloaded a callback function can now be called.
The callback function I call checks to see if the resolution of the image appears in the collection of resolutions that has been entered in the configuration file and deletes/moves accordingly.
If a file can not be read (which I have noticed happens sometimes), it is removed, not copied and not archived so that it can be retried later.
4grab got a new command-line option, -s --sorter, to sort out old images, running python sorter.py has the same effect, but this seemed pretties.
theoretically multiple categories could now be entered into the configuration file seperated by ',', but this hasn't been tested yet.
mutliple resolutions could be entered into the configuration file, seperated by ',' like so: 1680x1050,1920x1200.
Configuration now checks to see if all the necessary properties are available in the configuration file, if one is missing, it tries to create it.
2010-03-19 00:18:04 +01:00
|
|
|
|
2010-04-22 22:24:38 +02:00
|
|
|
if self.check_filename(filename):
|
Sorting, multi category, multi resolution
After a file has been downloaded a callback function can now be called.
The callback function I call checks to see if the resolution of the image appears in the collection of resolutions that has been entered in the configuration file and deletes/moves accordingly.
If a file can not be read (which I have noticed happens sometimes), it is removed, not copied and not archived so that it can be retried later.
4grab got a new command-line option, -s --sorter, to sort out old images, running python sorter.py has the same effect, but this seemed pretties.
theoretically multiple categories could now be entered into the configuration file seperated by ',', but this hasn't been tested yet.
mutliple resolutions could be entered into the configuration file, seperated by ',' like so: 1680x1050,1920x1200.
Configuration now checks to see if all the necessary properties are available in the configuration file, if one is missing, it tries to create it.
2010-03-19 00:18:04 +01:00
|
|
|
image = None
|
|
|
|
try:
|
|
|
|
image = Image.open(os.path.join(download_base,
|
|
|
|
filename))
|
|
|
|
except IOError:
|
2010-03-19 16:08:39 +01:00
|
|
|
retval = False
|
Sorting, multi category, multi resolution
After a file has been downloaded a callback function can now be called.
The callback function I call checks to see if the resolution of the image appears in the collection of resolutions that has been entered in the configuration file and deletes/moves accordingly.
If a file can not be read (which I have noticed happens sometimes), it is removed, not copied and not archived so that it can be retried later.
4grab got a new command-line option, -s --sorter, to sort out old images, running python sorter.py has the same effect, but this seemed pretties.
theoretically multiple categories could now be entered into the configuration file seperated by ',', but this hasn't been tested yet.
mutliple resolutions could be entered into the configuration file, seperated by ',' like so: 1680x1050,1920x1200.
Configuration now checks to see if all the necessary properties are available in the configuration file, if one is missing, it tries to create it.
2010-03-19 00:18:04 +01:00
|
|
|
|
|
|
|
if not image == None and self.archive_check(filename):
|
2010-04-22 22:24:38 +02:00
|
|
|
util.log(util.LogType.Msg, "Checking resolution", {"filename":filename, "resolution":image.size})
|
|
|
|
|
Sorting, multi category, multi resolution
After a file has been downloaded a callback function can now be called.
The callback function I call checks to see if the resolution of the image appears in the collection of resolutions that has been entered in the configuration file and deletes/moves accordingly.
If a file can not be read (which I have noticed happens sometimes), it is removed, not copied and not archived so that it can be retried later.
4grab got a new command-line option, -s --sorter, to sort out old images, running python sorter.py has the same effect, but this seemed pretties.
theoretically multiple categories could now be entered into the configuration file seperated by ',', but this hasn't been tested yet.
mutliple resolutions could be entered into the configuration file, seperated by ',' like so: 1680x1050,1920x1200.
Configuration now checks to see if all the necessary properties are available in the configuration file, if one is missing, it tries to create it.
2010-03-19 00:18:04 +01:00
|
|
|
for resolution in self.resolutions:
|
|
|
|
resolution = resolution.split('x')
|
|
|
|
foldername = "%s-%s" % (resolution[0],
|
|
|
|
resolution[1])
|
|
|
|
folderpath = os.path.join(download_base,
|
|
|
|
foldername)
|
|
|
|
|
|
|
|
if str(image.size[0]) == resolution[0] and \
|
|
|
|
str(image.size[1]) == resolution[1]:
|
|
|
|
if not os.path.exists(folderpath):
|
|
|
|
os.makedirs(folderpath)
|
|
|
|
|
|
|
|
self.copy(filename, folderpath)
|
|
|
|
break
|
2010-04-07 23:30:59 +02:00
|
|
|
self.archive(filename)
|
2010-04-19 09:16:04 +02:00
|
|
|
self.remove(filename)
|
|
|
|
return retval
|
Sorting, multi category, multi resolution
After a file has been downloaded a callback function can now be called.
The callback function I call checks to see if the resolution of the image appears in the collection of resolutions that has been entered in the configuration file and deletes/moves accordingly.
If a file can not be read (which I have noticed happens sometimes), it is removed, not copied and not archived so that it can be retried later.
4grab got a new command-line option, -s --sorter, to sort out old images, running python sorter.py has the same effect, but this seemed pretties.
theoretically multiple categories could now be entered into the configuration file seperated by ',', but this hasn't been tested yet.
mutliple resolutions could be entered into the configuration file, seperated by ',' like so: 1680x1050,1920x1200.
Configuration now checks to see if all the necessary properties are available in the configuration file, if one is missing, it tries to create it.
2010-03-19 00:18:04 +01:00
|
|
|
|
|
|
|
def copy(self, filename, destpath):
|
|
|
|
download_base = self.conf.get_download_location()
|
|
|
|
source = os.path.join(download_base,
|
|
|
|
os.path.basename(filename))
|
2010-03-22 15:17:55 +01:00
|
|
|
|
|
|
|
today = datetime.date.today()
|
Sorting, multi category, multi resolution
After a file has been downloaded a callback function can now be called.
The callback function I call checks to see if the resolution of the image appears in the collection of resolutions that has been entered in the configuration file and deletes/moves accordingly.
If a file can not be read (which I have noticed happens sometimes), it is removed, not copied and not archived so that it can be retried later.
4grab got a new command-line option, -s --sorter, to sort out old images, running python sorter.py has the same effect, but this seemed pretties.
theoretically multiple categories could now be entered into the configuration file seperated by ',', but this hasn't been tested yet.
mutliple resolutions could be entered into the configuration file, seperated by ',' like so: 1680x1050,1920x1200.
Configuration now checks to see if all the necessary properties are available in the configuration file, if one is missing, it tries to create it.
2010-03-19 00:18:04 +01:00
|
|
|
dest = os.path.join(destpath,
|
2010-03-22 15:17:55 +01:00
|
|
|
"%d-%d-%d" % (today.year, today.month, today.day))
|
2010-04-22 22:24:38 +02:00
|
|
|
util.log(util.LogType.Msg, "going to copy %s to %s" % (source, dest), None)
|
|
|
|
|
2010-03-22 15:17:55 +01:00
|
|
|
if not os.path.exists(dest):
|
|
|
|
os.makedirs(dest)
|
|
|
|
|
|
|
|
dest = os.path.join(dest,
|
Sorting, multi category, multi resolution
After a file has been downloaded a callback function can now be called.
The callback function I call checks to see if the resolution of the image appears in the collection of resolutions that has been entered in the configuration file and deletes/moves accordingly.
If a file can not be read (which I have noticed happens sometimes), it is removed, not copied and not archived so that it can be retried later.
4grab got a new command-line option, -s --sorter, to sort out old images, running python sorter.py has the same effect, but this seemed pretties.
theoretically multiple categories could now be entered into the configuration file seperated by ',', but this hasn't been tested yet.
mutliple resolutions could be entered into the configuration file, seperated by ',' like so: 1680x1050,1920x1200.
Configuration now checks to see if all the necessary properties are available in the configuration file, if one is missing, it tries to create it.
2010-03-19 00:18:04 +01:00
|
|
|
os.path.basename(filename))
|
2010-03-22 15:17:55 +01:00
|
|
|
|
Sorting, multi category, multi resolution
After a file has been downloaded a callback function can now be called.
The callback function I call checks to see if the resolution of the image appears in the collection of resolutions that has been entered in the configuration file and deletes/moves accordingly.
If a file can not be read (which I have noticed happens sometimes), it is removed, not copied and not archived so that it can be retried later.
4grab got a new command-line option, -s --sorter, to sort out old images, running python sorter.py has the same effect, but this seemed pretties.
theoretically multiple categories could now be entered into the configuration file seperated by ',', but this hasn't been tested yet.
mutliple resolutions could be entered into the configuration file, seperated by ',' like so: 1680x1050,1920x1200.
Configuration now checks to see if all the necessary properties are available in the configuration file, if one is missing, it tries to create it.
2010-03-19 00:18:04 +01:00
|
|
|
if source != dest:
|
|
|
|
shutil.copy(source, dest)
|
|
|
|
else:
|
|
|
|
print "\nHow can this even happen?! Copying", source, "to", dest
|
|
|
|
|
|
|
|
def archive(self, filename):
|
2010-04-19 09:16:04 +02:00
|
|
|
be = backend.Backend()
|
|
|
|
be.add(os.path.basename(filename))
|
Sorting, multi category, multi resolution
After a file has been downloaded a callback function can now be called.
The callback function I call checks to see if the resolution of the image appears in the collection of resolutions that has been entered in the configuration file and deletes/moves accordingly.
If a file can not be read (which I have noticed happens sometimes), it is removed, not copied and not archived so that it can be retried later.
4grab got a new command-line option, -s --sorter, to sort out old images, running python sorter.py has the same effect, but this seemed pretties.
theoretically multiple categories could now be entered into the configuration file seperated by ',', but this hasn't been tested yet.
mutliple resolutions could be entered into the configuration file, seperated by ',' like so: 1680x1050,1920x1200.
Configuration now checks to see if all the necessary properties are available in the configuration file, if one is missing, it tries to create it.
2010-03-19 00:18:04 +01:00
|
|
|
|
|
|
|
def archive_check(self, filename):
|
2010-04-19 09:16:04 +02:00
|
|
|
be = backend.Backend()
|
|
|
|
return be.check(os.path.basename(filename))
|
Sorting, multi category, multi resolution
After a file has been downloaded a callback function can now be called.
The callback function I call checks to see if the resolution of the image appears in the collection of resolutions that has been entered in the configuration file and deletes/moves accordingly.
If a file can not be read (which I have noticed happens sometimes), it is removed, not copied and not archived so that it can be retried later.
4grab got a new command-line option, -s --sorter, to sort out old images, running python sorter.py has the same effect, but this seemed pretties.
theoretically multiple categories could now be entered into the configuration file seperated by ',', but this hasn't been tested yet.
mutliple resolutions could be entered into the configuration file, seperated by ',' like so: 1680x1050,1920x1200.
Configuration now checks to see if all the necessary properties are available in the configuration file, if one is missing, it tries to create it.
2010-03-19 00:18:04 +01:00
|
|
|
|
|
|
|
def check_filename(self, filename):
|
|
|
|
ext = os.path.splitext(filename)[1]
|
|
|
|
return ext == ".jpg" or \
|
|
|
|
ext == ".png" or \
|
|
|
|
ext == ".gif"
|
|
|
|
|
|
|
|
def remove(self, filename):
|
|
|
|
download_base = self.conf.get_download_location()
|
|
|
|
source = os.path.join(download_base, filename)
|
|
|
|
os.remove(source)
|
|
|
|
|
|
|
|
if __name__ == "__main__":
|
|
|
|
conf = config.Configuration()
|
|
|
|
download_base = conf.get_download_location()
|
|
|
|
sorter = Sorter()
|
|
|
|
for item in os.listdir(download_base):
|
|
|
|
sorter.act(item)
|