2035 lines
92 KiB
Python
2035 lines
92 KiB
Python
"""Contains classes for a plain-text, client-server dbms.
|
|
|
|
Classes:
|
|
KirbyBase - database class
|
|
KBError - exceptions
|
|
|
|
Example:
|
|
from db import *
|
|
|
|
db = KirbyBase()
|
|
db.create('plane.tbl', ['name:str', 'country:str', 'speed:int',
|
|
'range:int'])
|
|
db.insert('plane.tbl', ['P-51', 'USA', 403, 1201])
|
|
db.insert('plane.tbl', ['P-38', 'USA', 377, 999])
|
|
db.select('plane.tbl', ['country', 'speed'], ['USA', '>400'])
|
|
db.update('plane.tbl', ['country'], ['USA'], ['United States'],
|
|
['country'])
|
|
db.delete('plane.tbl', ['speed'], ['<400'])
|
|
db.close()
|
|
|
|
Author:
|
|
Jamey Cribbs -- jcribbs@twmi.rr.com
|
|
www.netpromi.com
|
|
|
|
History:
|
|
2003-04-08: Version 1.0 released.
|
|
2003-04-15: Version 1.01 released.
|
|
-Added getFieldNames method.
|
|
-Added getFieldTypes method.
|
|
-Added drop method.
|
|
-Added check for existing file in create method.
|
|
-Improved documentation.
|
|
2003-07-16: Version 1.02 released.
|
|
-KirbyBase now uses distutils for installation.
|
|
-Fixed bug in createTable where field list was getting modified.
|
|
-Fixed bug in insert where values list was getting modified.
|
|
-Added the ability to use special characters in table data such
|
|
as \r, \n, \032, and |.
|
|
-Changed the name of all private methods to start with two
|
|
underscores, thereby following recommended naming conventions.
|
|
-Improved documentation.
|
|
-Added licensing information.
|
|
2003-08-14: Version 1.3 released.
|
|
-Added len method.
|
|
-Fixed bug in validateMatchCriteria where script was not
|
|
restricting other match criteria if already attempting to match
|
|
by recno.
|
|
-Fixed bug in validateMatchCriteria where script was not checking
|
|
to see if pattern argument was an integer when attempting to
|
|
match by recno.
|
|
-Added ability to pass field values to update and insert using a
|
|
dictionary.
|
|
-Added ability to specify field to sort on and sort direction for
|
|
the results of a select.
|
|
-Changed the way field types are handled internally. Instead of
|
|
treating them as strings (which is how they are stored) and
|
|
having to constantly 'eval' them to get the type, I decided to
|
|
work with them in their 'native' format. This should not change
|
|
any of the api or interfaces, EXCEPT for the getFieldTypes
|
|
method, which now returns a list of types, instead of a list of
|
|
strings. I hope this doesn't screw anyone's programs up.
|
|
-Corrected version number to conform to guidelines in distutils
|
|
documentation.
|
|
2003-08-27: Version 1.4 released.
|
|
-Added two new database field types: datetime.date and
|
|
datetime.datetime. They are stored as strings, but are input and
|
|
output as instances of datetime.date and datetime.datetime
|
|
respectively.
|
|
-Made a few internal optimizations when running queries that have
|
|
resulted in a 15-20% speed increase when doing large queries or
|
|
updates.
|
|
-Changed the name of all private methods from starting with two
|
|
underscores to starting with one underscore based on a discussion
|
|
in comp.lang.python as to how to properly name private variables.
|
|
2003-09-02: Version 1.5 released.
|
|
-Changed the way queries are handled internally. Instead of doing
|
|
an eval to do numeric and datetime comparisons, I changed it to do
|
|
the actual comparison itself. This resulted in a 40% speed
|
|
increase on large queries that do comparison expressions.
|
|
-Changed how data is passed between the server and the client in
|
|
client/server mode. I now use cPickle instead of repr and eval.
|
|
This resulted in an approximately 40% speed increase in
|
|
client/server operations.
|
|
2004-06-10: Version 1.5.1 released.
|
|
-Added a new database field type: boolean.
|
|
-Fixed a bug where KirbyBase was trying to convert an empty table
|
|
field (i.e. '') back into it's native format such as int or float
|
|
and raising an Exception. Now, if a table field is empty, I just
|
|
append it to the result record as is and dont' try to convert it.
|
|
-getMatches method will now split each database record only up to
|
|
the number of fields required to satisfy the query. This should
|
|
save a little query time on large databases with many fields.
|
|
-Added ability to have the getMatches method match string fields
|
|
based on string equality rather than regular expressions.
|
|
2004-06-22: Version 1.6 released.
|
|
-On numeric comparisons, you can now specify negative numbers.
|
|
-Fixed a bug where program would crash if you had a space between
|
|
the comparison operator and the number in numeric comparisons in
|
|
select statement.
|
|
-You can select all records by specifying that you want to match
|
|
'recno' against '*'.
|
|
-Got rid of the last eval in the code.
|
|
-Modest speed improvement by checking for strings that need to be
|
|
encoded first insteading of just encoding all strings.
|
|
-Added a compatibility layer for declaring field types when
|
|
creating a table. If you use the new compatible field types when
|
|
creating a table, you can use the table either from the Python or
|
|
Ruby version of KirbyBase without having to change anything.
|
|
-Changed the record-counter and deleted-records-counter in the
|
|
header record of the table to be zero padded instead of spaces
|
|
padded.
|
|
2005-01-30: Version 1.7 released.
|
|
-***IMPORTANT***
|
|
Changed the default value for the keyword argument 'useRegExp' to
|
|
be false instead of true. This means that, when doing a update,
|
|
delete, or select, records being selected on string fields will
|
|
be matched using exact matching instead of regular expression
|
|
matching. If you want to do regular expression matching, pass
|
|
'useRegExp = True' to the method.
|
|
-Added a new public method called validate. Calling this method
|
|
with a table name will check each record of that table and
|
|
validate that all of the fields have values of the correct type.
|
|
This can be used to validate data you have put into the table by
|
|
means other than through KirbyBase, perhaps by opening the table
|
|
in a text editor and typing in information.
|
|
-Fixed a bug in _closeTable where if an exception occurred it was
|
|
blowing up because the variable 'name' did not exist.
|
|
-Fixed a bug in _writeRecord where if an exception occured it was
|
|
blowing up because the variable 'name' did not exist.
|
|
-Fixed a bug in _getMatches where I was referencing
|
|
self.field_names as a method instead of as a dictionary.
|
|
-Added a keyword argument to select() called returnType. If set to
|
|
'object', the result list returned will contain Record objects
|
|
where each field name is an attribute of the object, so you could
|
|
refer to a record's field as plane.speed instead of plane[4]. If
|
|
set to 'dict', the result list returned will contain dictionaries
|
|
where each key is a field name and each value is a field value.
|
|
If set to 'list', the default, the result is a list of lists.
|
|
-Added a new method, insertBatch. It allows you to insert multiple
|
|
records at one time into a table.
|
|
-Added a new private method, _strToBool, that converts string
|
|
values like 'True' to boolean values.
|
|
-Added a new private method, _convertInput, and moved to it the
|
|
code that ensures that the data on an insert is in proper list
|
|
format. I did this so that I did not have duplicate code in both
|
|
the insert and insertBatch methods.
|
|
-To accomodate the fact that users can now send a large batch of
|
|
records to be inserted, I changed _sendSocket so that it first
|
|
sends the length of the database command to the server, then it
|
|
actually sends the command itself, which can now be any length.
|
|
-Changed the code in _getMatches to precompile the regular
|
|
expression pattern instead of dynamically compiling every time the
|
|
pattern is compared to a table record. This should speed up
|
|
queries a little bit.
|
|
-Changed the code in select that converts table fields back to
|
|
their native types to be more efficient.
|
|
-Changed _sendSocket to use StringIO (actually cStringIO) to hold
|
|
the result set of a client/server-based query instead of just
|
|
capturing the result by concatenating records to one big string.
|
|
In informal testing on large result sets, it shaves a few tenths
|
|
of a second off the query time.
|
|
2005-01-31: Version 1.7.1 released.
|
|
-Fixed a nasty bug in _getMatches. If useRegExp was True and the
|
|
select was against multiple string fields, the code was only
|
|
using one of the input patterns to search on instead of using all
|
|
of them.
|
|
2005-02-20: Version 1.8 released.
|
|
******** IMPORTANT - Method Interface Changes ****************
|
|
-Added the ability to sort the result set of a select by multiple
|
|
fields and to specify whether each field should be sorted
|
|
ascending or descending. This necessitated a change to the
|
|
interface of the select method. I moved the position of sortField
|
|
in the argument list and also changed it to be a list instead of a
|
|
string. I also changed the name of sortField to sortFields. I
|
|
also moved sortDesc in the arguement list and also made it a list.
|
|
******************************************************************
|
|
-Added another allowable value, 'report', to the keyword
|
|
parameter, returnType in the select method. This returns the
|
|
result set in a pretty print format. Along with this, added
|
|
another keyword parameter called rptSettings to the select method.
|
|
This is only used if rptType is 'report'. It is a 2 element list.
|
|
The first element specifies the number of records to print on a
|
|
page. The second element is boolean specifying whether to print
|
|
a dashed line between records.
|
|
-Added ability to pass field values to update and insert using an
|
|
object with attributes set equal to the field names.
|
|
-Fixed a bug in _getMatches. If a field of type int or float had
|
|
a blank value in the table (i.e. ''), the code was attempting to
|
|
convert it to it's proper type (i.e. int or float) before
|
|
doing the match comparison. This would cause an exception to
|
|
occur. Now, if the field is an int,float,date or datetime and it
|
|
is blank, I convert it to None. This allows the numeric
|
|
comparisons to work correctly for null fields.
|
|
-Fixed a bug in select. If a field in the result set was equal
|
|
to '', I was letting it stay that way, when I should really be
|
|
converting it to None before returning the result set.
|
|
-Cleaned up the internals a bit. Mainly, I tried to use functions
|
|
in the operator module like lt and ge instead of hardcoding < and
|
|
>= in an if statement.
|
|
2005-04-26: Version 1.8.1 released.
|
|
-Added the ability to select, update, delete multiple records by
|
|
specifying a list of record numbers.
|
|
-Cleaned up the internals of _getMatchesByRecno by not splitting
|
|
the record line into fields if user is selecting all records, and
|
|
by using a generator to return matches instead of building a list
|
|
of matches and returning the whole list.
|
|
2005-05-03: Version 1.8.2 released.
|
|
-Emergency bug-fix release: Fred Pacquier's eagle eyes spotte a bug
|
|
I introducted in version 1.8.1. It's in _getMatchByRecno. If you
|
|
are selecting all records, KirbyBase is not getting rid of the
|
|
newline character attached to the last field of the record.
|
|
Thanks Fred Pacquier!
|
|
2005-09-19: Version 1.9 released.
|
|
-Fixed a bug that can occur on very fast machines when sending data
|
|
between the client and the server. Thanks to David Edwards for
|
|
this fix.
|
|
-Added a method, setDefaultReturnType, that allows you to, set the
|
|
default return type of results from the select method. The default
|
|
is still 'list', but you can set it to 'list', 'object', or 'dict'.
|
|
Thanks to David Edwards for this suggested enhancement.
|
|
-Added methods addFields and dropFields. These allow you to add new
|
|
columns to a table and remove existing columns from the table.
|
|
Thanks to Pierre Quentel for the code for these two enhancements.
|
|
"""
|
|
import re
|
|
import socket
|
|
import os.path
|
|
import datetime
|
|
import cPickle
|
|
import cStringIO
|
|
import operator
|
|
import tempfile
|
|
import shutil
|
|
|
|
|
|
#--------------------------------------------------------------------------
|
|
# KirbyBase Class
|
|
#--------------------------------------------------------------------------
|
|
class KirbyBase:
|
|
"""Database Management System.
|
|
|
|
Public Methods:
|
|
__init__ - Create an instance of database.
|
|
close - Close database.
|
|
create - Create a table.
|
|
insert - Insert a record into a table.
|
|
insertBatch - Insert a list of records into a table.
|
|
update - Update a table.
|
|
delete - Delete record(s) from a table.
|
|
select - select record(s) from a table.
|
|
pack - remove deleted records from a table.
|
|
validate - validate data in table records.
|
|
drop - Remove a table.
|
|
getFieldNames - Get a list of a table's field names.
|
|
getFieldTypes - Get a list of a table's field types.
|
|
addFields - Insert new column(s) into table.
|
|
dropFields - Remove column(s) from table.
|
|
len - Total number of records in table.
|
|
setDefaultReturnType - Set the default return type for selects.
|
|
"""
|
|
|
|
#----------------------------------------------------------------------
|
|
# PUBLIC METHODS
|
|
#----------------------------------------------------------------------
|
|
|
|
#----------------------------------------------------------------------
|
|
# init
|
|
#----------------------------------------------------------------------
|
|
def __init__(self, type='local', host=None, port=None):
|
|
"""Create an instance of the database and return a reference to it.
|
|
|
|
Keyword Arguments:
|
|
type - Connection type: local(default), client, or server.
|
|
host - IP address of server to connect to, if connection type
|
|
is client.
|
|
port - Port number of server to connect to, if connection type
|
|
is client.
|
|
"""
|
|
self.connect_type = type
|
|
|
|
# Regular expression used to determine if field needs to be
|
|
# encoded.
|
|
self.encodeRegExp = re.compile(r'\n|\r|\032|\|')
|
|
|
|
# Regular expression used to determine if field needs to be
|
|
# un-encoded.
|
|
self.unencodeRegExp = re.compile(
|
|
r'&linefeed;|&carriage_return;|&substitute;|&pipe;')
|
|
|
|
# This will be used to validate the select statements.
|
|
self.cmpFuncs = {"<":operator.lt, "<=":operator.le,
|
|
">=":operator.ge, ">":operator.gt, "==":operator.eq,
|
|
"!=":operator.ne, "<>":operator.ne}
|
|
|
|
# This will be used to validate and convert the field types in
|
|
# the header rec of the table into valid python types.
|
|
self.strToTypes = {'int':int, 'Integer':int, 'float':float,
|
|
'Float':float, 'datetime.date':datetime.date,
|
|
'Date':datetime.date, 'datetime.datetime':datetime.datetime,
|
|
'DateTime':datetime.datetime, 'bool':bool, 'Boolean':bool,
|
|
'str':str, 'String':str}
|
|
|
|
# If connecting as client, open a socket connection with server.
|
|
if self.connect_type == 'client':
|
|
self.host, self.port = host, port
|
|
|
|
# Default select return type to list.
|
|
self.def_return_type = 'list'
|
|
|
|
#----------------------------------------------------------------------
|
|
# close
|
|
#----------------------------------------------------------------------
|
|
def close(self):
|
|
"""Close connection to database server.
|
|
"""
|
|
if self.connect_type == 'client':
|
|
self.dbSock.close()
|
|
|
|
#----------------------------------------------------------------------
|
|
# create
|
|
#----------------------------------------------------------------------
|
|
def create(self, name, fields):
|
|
"""Create a new table and return True on success.
|
|
|
|
Arguments:
|
|
name - physical filename, including path, that will hold
|
|
table.
|
|
fields - list holding strings made up of multiple fieldname,
|
|
fieldtype pairs (i.e. ['plane:str','speed:int']).
|
|
Valid fieldtypes are: str, int, float, datetime.date,
|
|
datetime.datetime, bool or, for compatibility with
|
|
the Ruby version of KirbyBase use String, Integer,
|
|
Float, Date, DateTime, and Boolean.
|
|
|
|
Returns True if no exceptions are raised.
|
|
"""
|
|
# If running as a client, then send the command to the server for
|
|
# it to execute.
|
|
if self.connect_type == 'client':
|
|
return self._sendSocket("db.create('%s',%s)" %(name,fields))
|
|
|
|
# Check to see if file already exists.
|
|
if os.path.exists(name):
|
|
raise KBError(name + ' already exists!')
|
|
|
|
# Validate field types. Integer, String, Float, Date, DateTime, and
|
|
# Boolean types are compatible between the Ruby and Python versions
|
|
# of KirbyBase.
|
|
for x in [y.split(':')[1] for y in fields]:
|
|
if x not in self.strToTypes:
|
|
raise KBError('Invalid field type: %s' % x)
|
|
|
|
# Make copy of fields list so that value passed in is not changed.
|
|
# Add recno counter, delete counter, and recno field definition at
|
|
# beginning.
|
|
header_rec = list(['000000','000000','recno:int'] + fields)
|
|
|
|
# Open the table in write mode since we are creating it new, write
|
|
# the header record to it and close it.
|
|
fptr = self._openTable(name, 'w')
|
|
fptr.write('|'.join(header_rec) + '\n')
|
|
self._closeTable(fptr)
|
|
|
|
# Return success.
|
|
return True
|
|
|
|
#----------------------------------------------------------------------
|
|
# insert
|
|
#----------------------------------------------------------------------
|
|
def insert(self, name, values):
|
|
"""Insert a new record into table, return unique record number.
|
|
|
|
Arguments:
|
|
name - physical file name, including path, that holds table.
|
|
values - list, dictionary, or object containing field values
|
|
of new record.
|
|
|
|
Returns unique record number assigned to new record when it is
|
|
created.
|
|
"""
|
|
# If running as a client, then send the command to the server for
|
|
# it to execute.
|
|
if self.connect_type == 'client':
|
|
# I put this code when I added the ability to use a record
|
|
# object for the values argument. The problem was that, if the
|
|
# user specified a record object, things worked ok in embedded
|
|
# mode. However, in client/server mode, I could not figure out
|
|
# a good way to send the record object to the server because it
|
|
# is embedded in a string representation of the database
|
|
# command (i.e. "db.insert('plane', recordObject)"). So, what
|
|
# I did to get it working was, I check to see if values is a
|
|
# record object. If it is, I grab the object's internal
|
|
# dictionary (__dict__) that holds all of the attributes and I
|
|
# send this as part of the expression string, instead of
|
|
# sending the object itself.
|
|
if not isinstance(values, (list, dict)):
|
|
return self._sendSocket("db.insert('%s',%s)" %(name,
|
|
values.__dict__))
|
|
else:
|
|
return self._sendSocket("db.insert('%s',%s)" %(name,values))
|
|
|
|
# Open the table.
|
|
fptr = self._openTable(name, 'r+')
|
|
|
|
# Update the instance variables holding table header info
|
|
self._updateHeaderVars(fptr)
|
|
|
|
# If values is a dictionary or an object, we are going to convert
|
|
# it into a list. That way, we can use the same validation and
|
|
# updating routines regardless of whether the user passed in a
|
|
# dictionary, an object, or a list. This returns a copy of
|
|
# values so that we are not messing with the original values.
|
|
record = self._convertInput(values)
|
|
|
|
# Check input fields to make sure they are valid.
|
|
self._validateUpdateCriteria(record, self.field_names[1:])
|
|
|
|
try:
|
|
# Get a new record number.
|
|
rec_no = self._incrRecnoCounter(fptr)
|
|
|
|
# Add record number to front of record.
|
|
record.insert(0, rec_no)
|
|
|
|
# Append the new record to the end of the table and close the
|
|
# table. Run each field through encoder to take care of
|
|
# special characters.
|
|
self._writeRecord(fptr, 'end', '|'.join(map(self._encodeString,
|
|
[str(item) for item in record])))
|
|
finally:
|
|
self._closeTable(fptr)
|
|
|
|
# Return the unique record number created for this new record.
|
|
return rec_no
|
|
|
|
#----------------------------------------------------------------------
|
|
# insertBatch
|
|
#----------------------------------------------------------------------
|
|
def insertBatch(self, name, batchRecords):
|
|
"""Insert a batch of records into table, return a list of rec #s.
|
|
|
|
Arguments:
|
|
name - physical file name, including path, that holds
|
|
table.
|
|
batchRecords - list of records. Each record can be a list, a
|
|
dictionary, or an object containing field values
|
|
of new record.
|
|
|
|
Returns list of unique record numbers assigned.
|
|
"""
|
|
# If running as a client, then send the command to the server for
|
|
# it to execute.
|
|
if self.connect_type == 'client':
|
|
return self._sendSocket("db.insertBatch('%s',%s)" %(name,
|
|
batchRecords))
|
|
|
|
# Open the table, update the instance variables holding table
|
|
# header info and close table.
|
|
fptr = self._openTable(name, 'r')
|
|
self._updateHeaderVars(fptr)
|
|
self._closeTable(fptr)
|
|
|
|
# Create an empty list to hold the batch after it has been
|
|
# validated and any records within it that are in dictionary format
|
|
# have been converted to list format.
|
|
records = []
|
|
|
|
for values in batchRecords:
|
|
# If values is a dictionary or an object, we are going to
|
|
# convert it into a list. That way, we can use the same
|
|
# validation and updating routines regardless of whether the
|
|
# user passed in a dictionary, an object, or a list. This
|
|
# returns a copy of values so that we are not messing with the
|
|
# original values.
|
|
record = self._convertInput(values)
|
|
# Check input fields to make sure they are valid.
|
|
self._validateUpdateCriteria(record, self.field_names[1:])
|
|
|
|
# Add the validated (and possibly converted) record to the
|
|
# records list.
|
|
records.append(record)
|
|
|
|
# Create empty list to hold new record numbers.
|
|
rec_nos = []
|
|
|
|
# Open the table again, this time in read-write mode.
|
|
fptr = self._openTable(name, 'r+')
|
|
|
|
try:
|
|
# Now that the batch has been validated, add it to the database
|
|
# table.
|
|
for record in records:
|
|
# Get a new record number.
|
|
rec_no = self._incrRecnoCounter(fptr)
|
|
|
|
# Add record number to front of record.
|
|
record.insert(0, rec_no)
|
|
|
|
# Append the new record to the end of the table. Run each
|
|
# field through encoder to take care of special characters.
|
|
self._writeRecord(fptr, 'end', '|'.join(
|
|
map(self._encodeString, [str(item) for item in record])))
|
|
|
|
# Add the newly create record number to the list of that we
|
|
# we return back to the user.
|
|
rec_nos.append(rec_no)
|
|
finally:
|
|
self._closeTable(fptr)
|
|
|
|
# Return the unique record number created for this new record.
|
|
return rec_nos
|
|
|
|
#----------------------------------------------------------------------
|
|
# update
|
|
#----------------------------------------------------------------------
|
|
def update(self, name, fields, searchData, updates, filter=None,
|
|
useRegExp=False):
|
|
"""Update record(s) in table, return number of records updated.
|
|
|
|
Arguments:
|
|
name - physical file name, including path, that holds
|
|
table.
|
|
fields - list containing names of fields to search on. If
|
|
any of the items in this list is 'recno', then the
|
|
table will be searched by the recno field only and
|
|
will update, at most, one record, since recno is
|
|
the system generated primary key.
|
|
searchData - list containing actual data to search on. Each
|
|
item in list corresponds to item in the 'fields'
|
|
list.
|
|
updates - list, dictionary, or object containing actual data
|
|
to put into table field. If it is a list and
|
|
'filter' list is empty or equal to None, then
|
|
updates list must have a value for each field in
|
|
table record.
|
|
filter - only used if 'updates' is a list. This is a
|
|
list containing names of fields to update. Each
|
|
item in list corresponds to item in the 'updates'
|
|
list. If 'filter' list is empty or equal to None,
|
|
and 'updates' is a list, then 'updates' list must
|
|
have an item for each field in table record,
|
|
excepting the recno field.
|
|
useRegExp - if true, match string fields using regular
|
|
expressions, else match string fields using
|
|
strict equality (i.e. '=='). Defaults to true.
|
|
|
|
Returns integer specifying number of records that were updated.
|
|
|
|
Example:
|
|
db.update('plane.tbl',['country','speed'],['USA','>400'],
|
|
[1230],['range'])
|
|
|
|
This will search for any plane from the USA with a speed
|
|
greater than 400mph and update it's range to 1230 miles.
|
|
"""
|
|
# If running as a client, then send the command to the server for
|
|
# it to execute.
|
|
if self.connect_type == 'client':
|
|
# I put this code when I added the ability to use a record
|
|
# object for the updates argument. The problem was that, if the
|
|
# user specified a record object, things worked ok in embedded
|
|
# mode. However, in client/server mode, I could not figure out
|
|
# a good way to send the record object to the server because it
|
|
# is embedded in a string representation of the database
|
|
# command (i.e. "db.update('plane', ['recno'], [45],
|
|
# recordObject)"). So, what I did to get it working was, I
|
|
# check to see if updates is a record object. If it is, I grab
|
|
# the object's internal dictionary (__dict__) that holds all of
|
|
# the attributes and I send this as part of the expression
|
|
# string, instead of sending the object itself.
|
|
if not isinstance(updates, (list, dict)):
|
|
return self._sendSocket("db.update('%s',%s,%s,%s,%s,%s)"
|
|
%(name, fields, searchData, updates.__dict__, filter,
|
|
useRegExp))
|
|
else:
|
|
return self._sendSocket("db.update('%s',%s,%s,%s,%s,%s)"
|
|
%(name, fields, searchData, updates, filter, useRegExp))
|
|
|
|
# Make copy of searchData list so that value passed in is not
|
|
# changed if I edit it in validateMatchCriteria.
|
|
patterns = list(searchData)
|
|
|
|
# Open the table.
|
|
fptr = self._openTable(name, 'r+')
|
|
|
|
# Update the instance variables holding table header info.
|
|
self._updateHeaderVars(fptr)
|
|
|
|
# If no update filter fields were specified, that means user wants
|
|
# to update all field in record, so we set the filter list equal
|
|
# to the list of field names of table, excluding the recno field,
|
|
# since user is not allowed to update recno field.
|
|
if filter:
|
|
if isinstance(updates, list):
|
|
pass
|
|
# If updates is a dictionary, user cannot specify a filter,
|
|
# because the keys of the dictionary will function as the
|
|
# filter.
|
|
elif isinstance(updates, dict):
|
|
raise KBError('Cannot specify filter when updates is a ' +
|
|
'dictionary.')
|
|
else:
|
|
raise KBError('Cannot specify filter when updates is an ' +
|
|
'object.')
|
|
|
|
else:
|
|
# If updates is a list and no update filter
|
|
# fields were specified, that means user wants to update
|
|
# all fields in record, so we set the filter list equal
|
|
# to the list of field names of table, excluding the recno
|
|
# field, since user is not allowed to update recno field.
|
|
if isinstance(updates, list): filter = self.field_names[1:]
|
|
|
|
# If updates is a list, do nothing because it is already in the
|
|
# proper format and filter has either been supplied by the user
|
|
# or populated above.
|
|
if isinstance(updates, list): pass
|
|
# If updates is a dictionary, we are going to convert it into an
|
|
# updates list and a filters list. This will allow us to use the
|
|
# same routines for validation and updating.
|
|
elif isinstance(updates, dict):
|
|
filter = [k for k in updates.keys() if k in
|
|
self.field_names[1:]]
|
|
updates = [updates[i] for i in filter]
|
|
# If updates is an object, we are going to convert it into an
|
|
# updates list and a filters list. This will allow us to use the
|
|
# same routines for validation and updating.
|
|
else:
|
|
filter = [x for x in self.field_names[1:] if hasattr(updates,x)]
|
|
updates = [getattr(updates,x) for x in self.field_names[1:] if
|
|
hasattr(updates,x)]
|
|
|
|
try:
|
|
# Check input arguments to make sure they are valid.
|
|
self._validateMatchCriteria(fields, patterns)
|
|
self._validateUpdateCriteria(updates, filter)
|
|
except KBError:
|
|
# If something didn't check out, close the table and re-raise
|
|
# the error.
|
|
fptr.close()
|
|
raise
|
|
|
|
# Search the table and populate the match list.
|
|
match_list = self._getMatches(fptr, fields, patterns, useRegExp)
|
|
|
|
# Create a list with each member being a list made up of a
|
|
# fieldname and the corresponding update value, converted to a
|
|
# safe string.
|
|
filter_updates = zip(filter,
|
|
[self._encodeString(str(u)) for u in updates])
|
|
|
|
updated = 0
|
|
# Step through the match list.
|
|
for line, fpos in match_list:
|
|
# Create a copy of the current record.
|
|
new_record = line.strip().split('|')
|
|
# For each filter field, apply the updated value to the
|
|
# table record.
|
|
for field, update in filter_updates:
|
|
new_record[self.field_names.index(field)] = update
|
|
|
|
# Convert the updated record back into a text line so we
|
|
# can write it back out to the file.
|
|
new_line = '|'.join(new_record)
|
|
|
|
# Since we are changing the current record, we will first
|
|
# write over it with all blank spaces in the file.
|
|
self._deleteRecord(fptr, fpos, line)
|
|
|
|
# If the updated copy of the record is not bigger than the
|
|
# old copy, then we can just write it in the same spot in
|
|
# the file. If it is bigger, then we will have to append
|
|
# it to the end of the file.
|
|
if len(new_line) > len(line):
|
|
self._writeRecord(fptr, 'end', new_line)
|
|
# If we didn't overwrite the current record, that means
|
|
# we have another blank record (i.e. delete record) out
|
|
# there, so we need to increment the deleted records
|
|
# counter.
|
|
self._incrDeleteCounter(fptr)
|
|
else:
|
|
self._writeRecord(fptr, fpos, new_line)
|
|
updated+=1
|
|
|
|
# Close the table.
|
|
self._closeTable(fptr)
|
|
|
|
# Return the number of records updated.
|
|
return updated
|
|
|
|
#----------------------------------------------------------------------
|
|
# delete
|
|
#----------------------------------------------------------------------
|
|
def delete(self, name, fields, searchData, useRegExp=False):
|
|
"""Delete record(s) from table, return number of records deleted.
|
|
|
|
Arguments:
|
|
name - physical file name, including path, that holds
|
|
table.
|
|
fields - list containing names of fields to search on. if
|
|
any of the items in this list is 'recno', then the
|
|
table will be searched by the recno field only and
|
|
will delete, at most, one record, since recno is
|
|
the system generated primary key.
|
|
searchData - list containing actual data to search on. Each
|
|
item in list corresponds to item in the 'fields'
|
|
list.
|
|
useRegExp - if true, match string fields using regular
|
|
expressions, else match string fields using
|
|
strict equality (i.e. '=='). Defaults to true.
|
|
|
|
Returns integer specifying number of records that were deleted.
|
|
|
|
Example:
|
|
db.delete('plane.tbl',['country','speed'],['USA','>400'])
|
|
|
|
This will search for any plane from the USA with a speed
|
|
greater than 400mph and delete it.
|
|
"""
|
|
# If running as a client, then send the command to the server for
|
|
# it to execute.
|
|
if self.connect_type == 'client':
|
|
return self._sendSocket("db.delete('%s',%s,%s,%s)"
|
|
%(name,fields, searchData, useRegExp))
|
|
|
|
# Make copy of searchData list so that value passed in is not
|
|
# changed if I edit it in validateMatchCriteria.
|
|
patterns = list(searchData)
|
|
|
|
# Open the table.
|
|
fptr = self._openTable(name, 'r+')
|
|
|
|
# Update the instance variables holding table header info.
|
|
self._updateHeaderVars(fptr)
|
|
|
|
try:
|
|
# Check input arguments to make sure they are valid.
|
|
self._validateMatchCriteria(fields, patterns)
|
|
except KBError:
|
|
# If something didn't check out, close the table and re-raise
|
|
# the error.
|
|
fptr.close()
|
|
raise
|
|
|
|
# Search the table and populate the match list.
|
|
match_list = self._getMatches(fptr, fields, patterns, useRegExp)
|
|
deleted = 0
|
|
|
|
# Delete any matches found.
|
|
for line, fpos in match_list:
|
|
self._deleteRecord(fptr, fpos, line)
|
|
# Increment the delete counter.
|
|
self._incrDeleteCounter(fptr)
|
|
deleted+=1
|
|
|
|
# Close the table.
|
|
self._closeTable(fptr)
|
|
|
|
# Return the number of records deleted.
|
|
return deleted
|
|
|
|
#----------------------------------------------------------------------
|
|
# select
|
|
#----------------------------------------------------------------------
|
|
def select(self, name, fields, searchData, filter=None,
|
|
useRegExp=False, sortFields=[], sortDesc=[], returnType=None,
|
|
rptSettings=[0,False]):
|
|
"""Select record(s) from table, return list of records selected.
|
|
|
|
Arguments:
|
|
name - physical file name, including path, that holds
|
|
table.
|
|
fields - list containing names of fields to search on.
|
|
If any of the items in this list is 'recno',
|
|
then the table will be searched by the recno
|
|
field only and will select, at most, one record,
|
|
since recno is the system generated primary key.
|
|
searchData - list containing actual data to search on. Each
|
|
item in list corresponds to item in the
|
|
'fields' list.
|
|
filter - list containing names of fields to include for
|
|
selected records. If 'filter' list is empty or
|
|
equal to None, then all fields will be included
|
|
in result set.
|
|
useRegExp - if true, match string fields using regular
|
|
expressions, else match string fields using
|
|
strict equality (i.e. '=='). Defaults to False.
|
|
sortFields - list of fieldnames to sort on. Each must be a
|
|
valid field name, and, if filter list is not
|
|
empty, the same fieldname must be in the filter
|
|
list. Result set will be sorted in the same
|
|
order as fields appear in sortFields in
|
|
ascending order unless the same field name also
|
|
appears in sortDesc, then they will be sorted in
|
|
descending order.
|
|
sortDesc - list of fieldnames that you want to sort result
|
|
set by in descending order. Each field name
|
|
must also be in sortFields.
|
|
returnType - a string specifying the type of the items in the
|
|
returned list. Can be 'list' (items are lists
|
|
of values), 'dict' (items are dictionaries with
|
|
keys = field names and values = matching
|
|
values), 'object' (items = instances of the
|
|
generic Record class) or 'report' (result set
|
|
is formatted as a table with a header, suitable
|
|
for printing). Defaults to None. If None, the
|
|
instance variable def_return_type will be used.
|
|
rptSettings - a list with two elements. This is only used if
|
|
returnType is 'report'. The first element
|
|
specifies the number of records to print on each
|
|
page. The default is 0, which means do not do
|
|
any page breaks. The second element is a
|
|
boolean specifying whether to print a row
|
|
separator (a line of dashes) between each
|
|
record. The default is False.
|
|
|
|
Returns list of records matching selection criteria.
|
|
|
|
Example:
|
|
db.select('plane.tbl',['country','speed'],['USA','>400'])
|
|
|
|
This will search for any plane from the USA with a speed
|
|
greater than 400mph and return it.
|
|
"""
|
|
# Check returnType argument to make sure it is either 'list',
|
|
# 'dict', or 'object'.
|
|
if returnType not in [None, 'list', 'dict', 'object', 'report']:
|
|
raise KBError('Invalid return type: %s' % returnType)
|
|
|
|
# If user did not specify a return type, use whatever the default
|
|
# return type is set to.
|
|
if returnType == None:
|
|
returnType = self.def_return_type
|
|
|
|
# Check rptSettings list to make sure it's items are valid.
|
|
if type(rptSettings[0]) != int:
|
|
raise KBError('Invalid report setting: %s' % rptSettings[0])
|
|
if type(rptSettings[1]) != bool:
|
|
raise KBError('Invalid report setting: %s' % rptSettings[1])
|
|
|
|
# If running as a client, then send the command to the server for
|
|
# it to execute.
|
|
if self.connect_type == 'client':
|
|
return self._sendSocket(
|
|
"db.select('%s',%s,%s,%s,%s,%s,%s,'%s',%s)"
|
|
%(name, fields, searchData, filter, useRegExp, sortFields,
|
|
sortDesc, returnType, rptSettings))
|
|
|
|
# Make copy of searchData list so that value passed in is not
|
|
# changed if I edit it in validateMatchCriteria.
|
|
patterns = list(searchData)
|
|
|
|
# Open the table in read-only mode since we won't be updating it.
|
|
fptr = self._openTable(name, 'r')
|
|
|
|
# Update the instance variables holding table header info.
|
|
self._updateHeaderVars(fptr)
|
|
|
|
try:
|
|
# Check input arguments to make sure they are valid.
|
|
self._validateMatchCriteria(fields, patterns)
|
|
if filter: self._validateFilter(filter)
|
|
else: filter = self.field_names
|
|
except KBError:
|
|
# If something didn't check out, close the table and re-raise
|
|
# the error.
|
|
fptr.close()
|
|
raise
|
|
|
|
# Validate sort field argument. It needs to be one of the field
|
|
# names included in the filter.
|
|
for field in [sf for sf in sortFields if sf not in filter]:
|
|
raise KBError('Invalid sort field specified: %s' % field)
|
|
# Make sure any fields specified in sort descending list are also
|
|
# in sort fields list.
|
|
for field in [sf for sf in sortDesc if sf not in sortFields]:
|
|
raise KBError('Cannot specify a field to sort in descending ' +
|
|
'order if you have not specified that field as a sort field')
|
|
|
|
# Search table and populate match list.
|
|
match_list = self._getMatches(fptr, fields, patterns, useRegExp)
|
|
|
|
# Initialize result set.
|
|
result_set = []
|
|
# Get a list of filter field indexes (i.e., where in the
|
|
# table record is the field that the filter item is
|
|
# referring to.
|
|
filterIndeces = [self.field_names.index(x) for x in filter]
|
|
|
|
# For each record in match list, add it to the result set.
|
|
for record, fpos in match_list:
|
|
# Initialize a record to hold the filtered fields of
|
|
# the record.
|
|
result_rec = []
|
|
# Split the record line into it's fields.
|
|
fields = record.split('|')
|
|
|
|
# Step through each field index in the filter list. Grab the
|
|
# result field at that position, convert it to
|
|
# proper type, and put it in result set.
|
|
for i in filterIndeces:
|
|
# If the field is empty, just append it to the result rec.
|
|
if fields[i] == '':
|
|
result_rec.append(None)
|
|
# Otherwise, convert field to its proper type before
|
|
# appending it to the result record.
|
|
else:
|
|
result_rec.append(
|
|
self.convert_types_functions[i](fields[i]))
|
|
|
|
# Add the result record to the result set.
|
|
result_set.append(result_rec)
|
|
|
|
# Close the table.
|
|
self._closeTable(fptr)
|
|
|
|
# If a sort field was specified...
|
|
# I stole the following code from Steve Lucy. I got it from the
|
|
# ASPN Python Cookbook webpages. Thanks Steve!
|
|
if len(sortFields) > 0:
|
|
reversedSortFields = list(sortFields)
|
|
reversedSortFields.reverse()
|
|
for sortField in reversedSortFields:
|
|
i = filter.index(sortField)
|
|
result_set.sort( lambda x,y:
|
|
cmp(*[(x[i], y[i]), (y[i], x[i])]
|
|
[sortField in sortDesc]))
|
|
|
|
# If returnType is 'object', then convert each result record
|
|
# to a Record object before returning the result list.
|
|
if returnType == 'object':
|
|
return [Record(filter, rec) for rec in result_set]
|
|
# If returnType is 'dict', then convert each result record to
|
|
# a dictionary with the keys being the field names before returning
|
|
# the result list.
|
|
elif returnType == 'dict':
|
|
return [dict(zip(filter, rec)) for rec in result_set]
|
|
# If returnType is 'report', then return a pretty print version of
|
|
# the result set.
|
|
elif returnType == 'report':
|
|
# How many records before a formfeed.
|
|
numRecsPerPage = rptSettings[0]
|
|
# Put a line of dashes between each record?
|
|
rowSeparator = rptSettings[1]
|
|
delim = ' | '
|
|
|
|
# columns of physical rows
|
|
columns = apply(zip, [filter] + result_set)
|
|
|
|
# get the maximum of each column by the string length of its
|
|
# items
|
|
maxWidths = [max([len(str(item)) for item in column])
|
|
for column in columns]
|
|
# Create a string of dashes the width of the print out.
|
|
rowDashes = '-' * (sum(maxWidths) + len(delim)*
|
|
(len(maxWidths)-1))
|
|
|
|
# select the appropriate justify method
|
|
justifyDict = {str:str.ljust,int:str.rjust,float:str.rjust,
|
|
bool:str.ljust,datetime.date:str.ljust,
|
|
datetime.datetime:str.ljust}
|
|
|
|
# Create a string that holds the header that will print.
|
|
headerLine = delim.join([justifyDict[fieldType](item,width)
|
|
for item,width,fieldType in zip(filter,maxWidths,
|
|
self.field_types)])
|
|
|
|
# Create a StringIO to hold the print out.
|
|
output=cStringIO.StringIO()
|
|
|
|
# Variable to hold how many records have been printed on the
|
|
# current page.
|
|
recsOnPageCount = 0
|
|
|
|
# For each row of the result set, print that row.
|
|
for row in result_set:
|
|
# If top of page, print the header and a dashed line.
|
|
if recsOnPageCount == 0:
|
|
print >> output, headerLine
|
|
print >> output, rowDashes
|
|
|
|
# Print a record.
|
|
print >> output, delim.join([justifyDict[fieldType](
|
|
str(item),width) for item,width,fieldType in
|
|
zip(row,maxWidths,self.field_types)])
|
|
|
|
# If rowSeparator is True, print a dashed line.
|
|
if rowSeparator: print >> output, rowDashes
|
|
|
|
# Add one to the number of records printed so far on
|
|
# the current page.
|
|
recsOnPageCount += 1
|
|
|
|
# If the user wants page breaks and you have printed
|
|
# enough records on this page, print a form feed and
|
|
# reset records printed variable.
|
|
if numRecsPerPage > 0 and (recsOnPageCount ==
|
|
numRecsPerPage):
|
|
print >> output, '\f',
|
|
recsOnPageCount = 0
|
|
# Return the contents of the StringIO.
|
|
return output.getvalue()
|
|
# Otherwise, just return the list of lists.
|
|
else:
|
|
return result_set
|
|
|
|
|
|
#----------------------------------------------------------------------
|
|
# pack
|
|
#----------------------------------------------------------------------
|
|
def pack(self, name):
|
|
"""Remove blank records from table and return total removed.
|
|
|
|
Keyword Arguments:
|
|
name - physical file name, including path, that holds table.
|
|
|
|
Returns number of blank lines removed from table.
|
|
"""
|
|
# If running as a client, then send the command to the server for
|
|
# it to execute.
|
|
if self.connect_type == 'client':
|
|
return self._sendSocket("db.pack('%s')" %(name))
|
|
|
|
# Open the table in read-only mode since we won't be updating it.
|
|
fptr = self._openTable(name, 'r')
|
|
|
|
# Read in all records.
|
|
lines = fptr.readlines()
|
|
|
|
# Close the table so we can re-build it.
|
|
self._closeTable(fptr)
|
|
|
|
# Reset number of deleted records to zero.
|
|
header_rec = lines[0].split('|')
|
|
header_rec[1] = "000000"
|
|
|
|
# Set first line of re-built file to the header record.
|
|
lines[0] = '|'.join(header_rec)
|
|
|
|
# Open the table in write mode since we will be re-building it.
|
|
fptr = self._openTable(name, 'w')
|
|
|
|
# This is the counter we will use to report back how many blank
|
|
# records were removed.
|
|
lines_deleted = 0
|
|
|
|
# Step through all records in table, only writing out non-blank
|
|
# records.
|
|
for line in lines:
|
|
# By doing a rstrip instead of a strip, we can remove any
|
|
# extra spaces at the end of line that were a result of
|
|
# updating a record with a shorter one.
|
|
line = line.rstrip()
|
|
if line == "":
|
|
lines_deleted += 1
|
|
continue
|
|
try:
|
|
fptr.write(line + '\n')
|
|
except:
|
|
raise KBError('Could not write record in: ' + name)
|
|
|
|
# Close the table.
|
|
self._closeTable(fptr)
|
|
|
|
# Return number of records removed from table.
|
|
return lines_deleted
|
|
|
|
#----------------------------------------------------------------------
|
|
# validate
|
|
#----------------------------------------------------------------------
|
|
def validate(self, name):
|
|
"""Validate all records have field values of proper data type.
|
|
|
|
Keyword Arguments:
|
|
name - physical file name, including path, that holds table.
|
|
|
|
Returns list of records that have invalid data.
|
|
"""
|
|
# If running as a client, then send the command to the server for
|
|
# it to execute.
|
|
if self.connect_type == 'client':
|
|
return self._sendSocket("db.validate('%s')" %(name))
|
|
|
|
# Open the table in read-only mode since we won't be updating it.
|
|
fptr = self._openTable(name, 'r')
|
|
|
|
# Update the instance variables holding table header info
|
|
self._updateHeaderVars(fptr)
|
|
|
|
# Create list to hold any invalid records that are found.
|
|
invalid_list = []
|
|
|
|
# Loop through all records in the table.
|
|
for line in fptr:
|
|
# Strip off newline character and any trailing spaces.
|
|
line = line[:-1].strip()
|
|
|
|
# If blank line, skip this record.
|
|
if line == "": continue
|
|
|
|
# Split the line up into fields.
|
|
record = line.split("|")
|
|
|
|
# Check the value of recno to see if the value is
|
|
# greater than the last recno assigned. If it is,
|
|
# add this to the invalid record list.
|
|
if self.last_recno < int(record[0]):
|
|
invalid_list.append([record[0], 'recno', record[0]])
|
|
|
|
# For each field in the record check to see if you
|
|
# can convert it to the field type specified in the
|
|
# header record by using the conversion function
|
|
# specified in self.convert_types_functions.
|
|
# If you can't convert it, add the
|
|
# record number, the field name, and the offending
|
|
# field value to the list of invalid records.
|
|
for i, item in enumerate(record):
|
|
if item == '': continue
|
|
try:
|
|
if self.convert_types_functions[i](item): pass
|
|
except:
|
|
invalid_list.append([record[0], self.field_names[i],
|
|
item])
|
|
|
|
# Close the table.
|
|
self._closeTable(fptr)
|
|
|
|
# Return list of invalid records.
|
|
return invalid_list
|
|
|
|
#----------------------------------------------------------------------
|
|
# drop
|
|
#----------------------------------------------------------------------
|
|
def drop(self, name):
|
|
"""Delete physical file containing table and return True.
|
|
|
|
Arguments:
|
|
name - physical filename, including path, that holds table.
|
|
|
|
Returns True if no exceptions are raised.
|
|
"""
|
|
# If running as a client, then send the command to the server for
|
|
# it to execute.
|
|
if self.connect_type == 'client':
|
|
return self._sendSocket("db.drop('%s')" %(name))
|
|
|
|
# Delete physical file.
|
|
os.remove(name)
|
|
|
|
# Return success.
|
|
return True
|
|
|
|
#----------------------------------------------------------------------
|
|
# getFieldNames
|
|
#----------------------------------------------------------------------
|
|
def getFieldNames(self, name):
|
|
"""Return list of field names for specified table name
|
|
|
|
Arguments:
|
|
name - physical file name, including path, that holds table.
|
|
|
|
Returns list of field names for table.
|
|
"""
|
|
# If running as a client, then send the command to the server for
|
|
# it to execute.
|
|
if self.connect_type == 'client':
|
|
return self._sendSocket("db.getFieldNames('%s')" %(name))
|
|
|
|
# Open the table in read-only mode since we won't be updating it.
|
|
fptr = self._openTable(name, 'r')
|
|
|
|
# Update the instance variables holding table header info
|
|
self._updateHeaderVars(fptr)
|
|
|
|
# Close the table.
|
|
self._closeTable(fptr)
|
|
|
|
return self.field_names
|
|
|
|
#----------------------------------------------------------------------
|
|
# getFieldTypes
|
|
#----------------------------------------------------------------------
|
|
def getFieldTypes(self, name):
|
|
"""Return list of field types for specified table name
|
|
|
|
Arguments:
|
|
name - physical file name, including path, that holds table.
|
|
|
|
Returns list of field types for table.
|
|
"""
|
|
# If running as a client, then send the command to the server for
|
|
# it to execute.
|
|
if self.connect_type == 'client':
|
|
return self._sendSocket("db.getFieldTypes('%s')" %(name))
|
|
|
|
# Open the table in read-only mode since we won't be updating it.
|
|
fptr = self._openTable(name, 'r')
|
|
|
|
# Update the instance variables holding table header info
|
|
self._updateHeaderVars(fptr)
|
|
|
|
# Close the table.
|
|
self._closeTable(fptr)
|
|
|
|
return self.field_types
|
|
|
|
#----------------------------------------------------------------------
|
|
# addFields
|
|
#----------------------------------------------------------------------
|
|
def addFields(self, name, fields, after = ''):
|
|
"""Modify the table to insert new fields after the specified field,
|
|
or at the beginning if after is None
|
|
|
|
Arguments:
|
|
name - physical file name, including path, that holds
|
|
table.
|
|
fields - list containing names of fields to insert.
|
|
after - name of the field after which the new fields
|
|
will be inserted, or empty string to insert at
|
|
the beginning of the record
|
|
|
|
A temporary file is created, then copied into the old file
|
|
"""
|
|
# If running as a client, then send the command to the server for
|
|
# it to execute.
|
|
if self.connect_type == 'client':
|
|
return self._sendSocket("db.addFields('%s',%s,'%s')" %(name,
|
|
fields,after))
|
|
|
|
# Validate field types
|
|
for x in [y.split(':')[1] for y in fields]:
|
|
if x not in self.strToTypes:
|
|
raise KBError('Invalid field type: %s' % x)
|
|
# Validate the field given in "after"
|
|
field_names = self.getFieldNames(name)
|
|
if after and after not in field_names:
|
|
raise KBError('Invalid field name: %s' % after)
|
|
# index to insert the new fields
|
|
if not after:
|
|
insert_after = 0
|
|
else:
|
|
insert_after = field_names.index(after)
|
|
# build the modified fields list
|
|
old_fields = zip(field_names,self.getFieldTypes(name))
|
|
new_fields = []
|
|
for (n,t) in old_fields[1:]:
|
|
if t.__name__ in ['date','datetime']:
|
|
new_fields.append('%s:datetime.%s' %(n,t.__name__))
|
|
else:
|
|
new_fields.append('%s:%s' %(n,t.__name__))
|
|
new_fields = new_fields[:insert_after]+fields+ \
|
|
new_fields[insert_after:]
|
|
|
|
# Use tempfile to get a unique name for a temporary file
|
|
# to insert the records of the modified table
|
|
temp_file=tempfile.NamedTemporaryFile()
|
|
temp_name=temp_file.name
|
|
temp_file.close()
|
|
|
|
# Recreate the header line to go into the temp file.
|
|
header_rec = list(['%06d' % self.last_recno,
|
|
'%06d' % self.del_counter, 'recno:int'] + new_fields)
|
|
|
|
# copy old table into temporary one
|
|
in_file = open(name,'r')
|
|
in_file.readline() # skip first line
|
|
out_file = open(temp_name,'a')
|
|
# Write header rec to new file.
|
|
out_file.write('|'.join(header_rec) + '\n')
|
|
|
|
blank = ['']*len(fields)
|
|
# build the new file faster than with selects from old file
|
|
# and inserts in new one
|
|
for line in in_file:
|
|
recs = line[:-1].split('|')
|
|
recs = recs[:insert_after+1]+blank+recs[insert_after+1:]
|
|
out_file.write('|'.join(recs)+'\n')
|
|
out_file.close()
|
|
# copy temporary file into old file
|
|
shutil.copyfile(temp_name,name)
|
|
# delete temporary file
|
|
os.remove(temp_name)
|
|
return True
|
|
|
|
#----------------------------------------------------------------------
|
|
# dropFields
|
|
#----------------------------------------------------------------------
|
|
def dropFields(self, name, fields, after = ''):
|
|
"""Modify the table to drop the specified fields
|
|
|
|
Arguments:
|
|
name - physical file name, including path, that holds
|
|
table.
|
|
fields - list containing names of fields to drop.
|
|
|
|
A temporary file is created, then copied into the old file
|
|
"""
|
|
# If running as a client, then send the command to the server for
|
|
# it to execute.
|
|
if self.connect_type == 'client':
|
|
return self._sendSocket("db.dropFields('%s',%s)" %(name,fields))
|
|
|
|
# Validate the fields
|
|
if 'recno' in fields:
|
|
raise KBError("Can't drop the field 'recno'")
|
|
field_names = self.getFieldNames(name)
|
|
for field in fields:
|
|
if field not in field_names:
|
|
raise KBError('Invalid field name : %s' %field)
|
|
# build the modified fields list
|
|
old_fields = zip(field_names,self.getFieldTypes(name))
|
|
new_fields = []
|
|
dropped_indeces = []
|
|
for i,(n,t) in enumerate(old_fields[1:]):
|
|
if not n in fields:
|
|
if t.__name__ in ['date','datetime']:
|
|
new_fields.append('%s:datetime.%s' %(n,t.__name__))
|
|
else:
|
|
new_fields.append('%s:%s' %(n,t.__name__))
|
|
else:
|
|
dropped_indeces.append(i+1)
|
|
|
|
# Use tempfile to get a unique name for a temporary file
|
|
# to insert the records of the modified table
|
|
temp_file=tempfile.NamedTemporaryFile()
|
|
temp_name=temp_file.name
|
|
temp_file.close()
|
|
|
|
# Recreate the header rec to go into the temp file.
|
|
header_rec = list(['%06d' % self.last_recno,
|
|
'%06d' % self.del_counter, 'recno:int'] + new_fields)
|
|
|
|
# copy old table into temporary one
|
|
in_file = open(name,'r')
|
|
in_file.readline() # skip first line
|
|
out_file = open(temp_name,'a')
|
|
# Write header rec to new file.
|
|
out_file.write('|'.join(header_rec) + '\n')
|
|
|
|
# build the new file faster than with selects from old file
|
|
# and inserts in new one
|
|
for line in in_file:
|
|
recs = line[:-1].split('|')
|
|
new_recs = [ item for (i,item) in enumerate(recs) \
|
|
if not i in dropped_indeces]
|
|
out_file.write('|'.join(new_recs)+'\n')
|
|
out_file.close()
|
|
# copy temporary file into old file
|
|
shutil.copyfile(temp_name,name)
|
|
# delete temporary file
|
|
os.remove(temp_name)
|
|
return True
|
|
|
|
#----------------------------------------------------------------------
|
|
# len
|
|
#----------------------------------------------------------------------
|
|
def len(self, name):
|
|
'''Return total number of non-deleted records in specified table
|
|
|
|
Arguments:
|
|
name - physical file name, including path, that holds table.
|
|
|
|
Returns total number of records in table.
|
|
'''
|
|
# If running as a client, then send the command to the server for
|
|
# it to execute.
|
|
if self.connect_type == 'client':
|
|
return self._sendSocket("db.len('%s')" %(name))
|
|
|
|
# Initialize counter.
|
|
rec_count = 0
|
|
|
|
# Open the table in read-only mode since we won't be updating it.
|
|
fptr = self._openTable(name, 'r')
|
|
|
|
# Skip header record.
|
|
line = fptr.readline()
|
|
|
|
# Loop through entire table.
|
|
line = fptr.readline()
|
|
while line:
|
|
# Strip off newline character.
|
|
line = line[0:-1]
|
|
|
|
# If not blank line, add 1 to record count.
|
|
if line.strip() != "": rec_count += 1
|
|
|
|
# Read next record.
|
|
line = fptr.readline()
|
|
|
|
# Close the table.
|
|
self._closeTable(fptr)
|
|
|
|
return rec_count
|
|
|
|
#----------------------------------------------------------------------
|
|
# setDefaultReturnType
|
|
#----------------------------------------------------------------------
|
|
def setDefaultReturnType(self, ret_type):
|
|
"""Set the default return type for selects.
|
|
"""
|
|
if ret_type not in ['list', 'dict', 'object', 'report']:
|
|
raise KBError('Invalid return type: %s' % ret_type)
|
|
self.def_return_type = ret_type
|
|
|
|
#----------------------------------------------------------------------
|
|
# PRIVATE METHODS
|
|
#----------------------------------------------------------------------
|
|
|
|
|
|
#----------------------------------------------------------------------
|
|
# _strToBool
|
|
#----------------------------------------------------------------------
|
|
def _strToBool(self, boolString):
|
|
if boolString == 'True': return True
|
|
elif boolString == 'False': return False
|
|
else: raise KBError('Invalid value for boolean: %s' % boolString)
|
|
|
|
|
|
#----------------------------------------------------------------------
|
|
# _strToDate
|
|
#----------------------------------------------------------------------
|
|
def _strToDate(self, dateString):
|
|
# Split the date string up into pieces and create a
|
|
# date object.
|
|
return datetime.date(*map(int, dateString.split('-')))
|
|
|
|
#----------------------------------------------------------------------
|
|
# _strToDateTime
|
|
#----------------------------------------------------------------------
|
|
def _strToDateTime(self, dateTimeString):
|
|
# Split datetime string into datetime portion microseconds portion.
|
|
tempDateTime = dateTimeString.split('.')
|
|
# Were there microseconds in the datetime string.
|
|
if len(tempDateTime) > 1: microsec = int(tempDateTime[1])
|
|
else: microsec = 0
|
|
|
|
# Now, split the datetime portion into a date
|
|
# and a time string. Take all of the pieces and
|
|
# create a datetime object.
|
|
tempDate, tempTime = tempDateTime[0].split(' ')
|
|
y, m, d = tempDate.split('-')
|
|
h, min, sec = tempTime.split(':')
|
|
return datetime.datetime(int(y),int(m),int(d),int(h),int(min),
|
|
int(sec),microsec)
|
|
|
|
#----------------------------------------------------------------------
|
|
# _encodeString
|
|
#----------------------------------------------------------------------
|
|
def _encodeString(self, s):
|
|
'''Encode a string.
|
|
|
|
Translates problem characters like \n, \r, and \032 to benign
|
|
character strings.
|
|
|
|
Keyword Arguments:
|
|
s - string to encode.
|
|
|
|
Returns encoded string.
|
|
'''
|
|
if self.encodeRegExp.search(s):
|
|
s = s.replace('\n', '&linefeed;')
|
|
s = s.replace('\r', '&carriage_return;')
|
|
s = s.replace('\032', '&substitute;')
|
|
s = s.replace('|', '&pipe;')
|
|
return s
|
|
|
|
#----------------------------------------------------------------------
|
|
# _unencodeString
|
|
#----------------------------------------------------------------------
|
|
def _unencodeString(self, s):
|
|
'''Unencode a string.
|
|
|
|
Translates encoded character strings back to special characters
|
|
like \n, \r, \032.
|
|
|
|
Keyword Arguments:
|
|
s - string to unencode.
|
|
|
|
Returns unencoded string.
|
|
'''
|
|
if self.unencodeRegExp.search(s):
|
|
s = s.replace('&linefeed;', '\n')
|
|
s = s.replace('&carriage_return;', '\r')
|
|
s = s.replace('&substitute;', '\032')
|
|
s = s.replace('&pipe;', '|')
|
|
return s
|
|
|
|
#----------------------------------------------------------------------
|
|
# _updateHeaderVars
|
|
#----------------------------------------------------------------------
|
|
def _updateHeaderVars(self, fptr):
|
|
# Go to the header record and read it in.
|
|
fptr.seek(0)
|
|
line = fptr.readline()
|
|
|
|
# Chop off the newline character.
|
|
line = line[0:-1]
|
|
|
|
# Split the record into fields.
|
|
header_rec = line.split('|')
|
|
|
|
# Update Last Record Number and Deleted Records counters.
|
|
self.last_recno = int(header_rec[0])
|
|
self.del_counter = int(header_rec[1])
|
|
|
|
# Skip the recno counter, and the delete counter.
|
|
header_fields = header_rec[2:]
|
|
|
|
# Create an instance variable holding the field names.
|
|
self.field_names = [item.split(':')[0] for item in header_fields]
|
|
# Create an instance variable holding the field types.
|
|
self.field_types = [self.strToTypes[x.split(':')[1]] for x in
|
|
header_fields]
|
|
|
|
# the functions to use to convert values as strings into their type
|
|
convTypes={int:int,float:float,bool:self._strToBool,
|
|
str:self._unencodeString,
|
|
datetime.date:self._strToDate,
|
|
datetime.datetime:self._strToDateTime}
|
|
self.convert_types_functions = [convTypes[f] for f in
|
|
self.field_types]
|
|
|
|
|
|
#----------------------------------------------------------------------
|
|
# _validateMatchCriteria
|
|
#----------------------------------------------------------------------
|
|
def _validateMatchCriteria(self, fields, patterns):
|
|
"""Run various checks against list of fields and patterns to be
|
|
used as search criteria. This method is called from all public
|
|
methods that search the database.
|
|
"""
|
|
if len(fields) == 0:
|
|
raise KBError('Length of fields list must be greater ' +
|
|
'than zero.')
|
|
if len(fields) != len(patterns):
|
|
raise KBError('Length of fields list and patterns list ' +
|
|
'not the same.')
|
|
|
|
# If any of the list of fields to search on do not match a field
|
|
# in the table, raise an error.
|
|
for field, pattern in zip(fields, patterns):
|
|
if field not in self.field_names:
|
|
raise KBError('Invalid field name in fields list: %s'
|
|
%field)
|
|
|
|
# If the field is recno, make sure they are trying not to
|
|
# search on more than one field. Also, make sure they are
|
|
# either trying to match all records or that it is an integer.
|
|
if field == 'recno':
|
|
if len(fields) > 1:
|
|
raise KBError('If selecting by recno, no other ' +
|
|
'selection criteria is allowed')
|
|
if pattern != '*':
|
|
# check if all specified recnos are integers
|
|
if not isinstance(pattern,(tuple,list)):
|
|
pattern = [pattern]
|
|
for x in pattern:
|
|
if not isinstance(x,int):
|
|
raise KBError('Recno argument %s has type %s'
|
|
', expected an integer' %(x,type(x)))
|
|
continue
|
|
|
|
# If the field type is not a str or a bool, make sure the
|
|
# pattern you are searching on has a proper comparion
|
|
# operator (<,<=,>,>=,==,!=,or <>) in it.
|
|
if (self.field_types[self.field_names.index(field)] in
|
|
[int, float, datetime.date, datetime.datetime]):
|
|
r = re.search('[\s]*[\+-]?\d', pattern)
|
|
if not self.cmpFuncs.has_key(pattern[:r.start()]):
|
|
raise KBError('Invalid comparison syntax: %s'
|
|
% pattern[:r.start()])
|
|
|
|
|
|
#----------------------------------------------------------------------
|
|
#_convertInput
|
|
#----------------------------------------------------------------------
|
|
def _convertInput(self, values):
|
|
"""If values is a dictionary or an object, we are going to convert
|
|
it into a list. That way, we can use the same validation and
|
|
updating routines regardless of whether the user passed in a
|
|
dictionary, an object, or a list.
|
|
"""
|
|
# If values is a list, make a copy of it.
|
|
if isinstance(values, list): record = list(values)
|
|
# If values is a dictionary, convert it's values into a list
|
|
# corresponding to the table's field names. If there is not
|
|
# a key in the dictionary corresponding to a field name, place a
|
|
# '' in the list for that field name's value.
|
|
elif isinstance(values, dict):
|
|
record = [values.get(k,'') for k in self.field_names[1:]]
|
|
# If values is a record object, then do the same thing for it as
|
|
# you would do for a dictionary above.
|
|
else:
|
|
record = [getattr(values,a,'') for a in self.field_names[1:]]
|
|
# Return the new list with all items == None replaced by ''.
|
|
new_rec = []
|
|
for r in record:
|
|
if r == None:
|
|
new_rec.append('')
|
|
else:
|
|
new_rec.append(r)
|
|
return new_rec
|
|
|
|
|
|
#----------------------------------------------------------------------
|
|
# _validateUpdateCriteria
|
|
#----------------------------------------------------------------------
|
|
def _validateUpdateCriteria(self, updates, filter):
|
|
"""Run various checks against list of updates and fields to be
|
|
used as update criteria. This method is called from all public
|
|
methods that update the database.
|
|
"""
|
|
if len(updates) == 0:
|
|
raise KBError('Length of updates list must be greater ' +
|
|
'than zero.')
|
|
|
|
if len(updates) != len(filter):
|
|
raise KBError('Length of updates list and filter list ' +
|
|
'not the same.')
|
|
# Since recno is the record's primary key and is system
|
|
# generated, like an autoincrement field, do not allow user
|
|
# to update it.
|
|
if 'recno' in filter:
|
|
raise KBError("Cannot update value of 'recno' field.")
|
|
|
|
# Validate filter list.
|
|
self._validateFilter(filter)
|
|
|
|
# Make sure each update is of the proper type.
|
|
for update, field_name in zip(updates, filter):
|
|
if update in ['', None]: pass
|
|
elif type(update) != self.field_types[
|
|
self.field_names.index(field_name)]:
|
|
raise KBError("Invalid update value for %s" % field_name)
|
|
|
|
#----------------------------------------------------------------------
|
|
# _validateFilter
|
|
#----------------------------------------------------------------------
|
|
def _validateFilter(self, filter):
|
|
# Each field in the filter list must be a valid field in the table.
|
|
for field in filter:
|
|
if field not in self.field_names:
|
|
raise KBError('Invalid field name: %s' % field)
|
|
|
|
#----------------------------------------------------------------------
|
|
# _getMatches
|
|
#----------------------------------------------------------------------
|
|
def _getMatches(self, fptr, fields, patterns, useRegExp):
|
|
# Initialize a list to hold all records that match the search
|
|
# criteria.
|
|
match_list = []
|
|
|
|
# If one of the fields to search on is 'recno', which is the
|
|
# table's primary key, then search just on that field and return
|
|
# at most one record.
|
|
if 'recno' in fields:
|
|
return self._getMatchByRecno(fptr,patterns)
|
|
# Otherwise, search table, using all search fields and patterns
|
|
# specified in arguments lists.
|
|
else:
|
|
new_patterns = []
|
|
fieldNrs = [self.field_names.index(x) for x in fields]
|
|
for fieldPos, pattern in zip(fieldNrs, patterns):
|
|
if self.field_types[fieldPos] == str:
|
|
# If useRegExp is True, compile the pattern to a
|
|
# regular expression object and add it to the
|
|
# new_patterns list. Otherwise, just add it to
|
|
# the new_patterns list. This will be used below
|
|
# when matching table records against the patterns.
|
|
if useRegExp:
|
|
new_patterns.append(re.compile(pattern))
|
|
# the pattern can be a tuple with re flags like re.I
|
|
else:
|
|
new_patterns.append(pattern)
|
|
elif self.field_types[fieldPos] == bool:
|
|
# If type is boolean, I am going to coerce it to be
|
|
# either True or False by applying bool to it. This
|
|
# is because it could be '' or []. Next, I am going
|
|
# to convert it to the string representation: either
|
|
# 'True' or 'False'. The reason I do this is because
|
|
# that is how it is stored in each record of the table
|
|
# and it is a lot faster to change this one value from
|
|
# boolean to string than to change possibly thousands
|
|
# of table values from string to boolean. And, if they
|
|
# both are either 'True' or 'False' I can still
|
|
# compare them using the equality test and get the same
|
|
# result as if they were both booleans.
|
|
new_patterns.append(str(bool(pattern)))
|
|
else:
|
|
# If type is int, float, date, or datetime, this next
|
|
# bit of code will split the the comparison string
|
|
# into the string representing the comparison
|
|
# operator (i.e. ">=" and the actual value we are going
|
|
# to compare the table records against from the input
|
|
# pattern, (i.e. "5"). So, for example, ">5" would be
|
|
# split into ">" and "5".
|
|
r = re.search('[\s]*[\+-]?\d', pattern)
|
|
if self.field_types[fieldPos] == int:
|
|
patternValue = int(pattern[r.start():])
|
|
elif self.field_types[fieldPos] == float:
|
|
patternValue = float(pattern[r.start():])
|
|
else:
|
|
patternValue = pattern[r.start():]
|
|
new_patterns.append(
|
|
[self.cmpFuncs[pattern[:r.start()]], patternValue]
|
|
)
|
|
|
|
fieldPos_new_patterns = zip(fieldNrs, new_patterns)
|
|
maxfield = max(fieldNrs)+1
|
|
|
|
# Record current position in table. Then read first detail
|
|
# record.
|
|
fpos = fptr.tell()
|
|
line = fptr.readline()
|
|
|
|
# Loop through entire table.
|
|
while line:
|
|
# Strip off newline character and any trailing spaces.
|
|
line = line[:-1].strip()
|
|
# Keep track of matching
|
|
is_no_match = False
|
|
# If blank line, skip this record.
|
|
if line == "": is_no_match = True
|
|
# Split the line up into fields.
|
|
record = line.split("|", maxfield)
|
|
|
|
if not is_no_match:
|
|
# Foreach correspond field and pattern, check to see
|
|
# if the table record's field matches successfully.
|
|
for fieldPos, pattern in fieldPos_new_patterns:
|
|
# If the field type is string, it
|
|
# must be an exact match or a regular expression,
|
|
# so we will compare the table record's field to it
|
|
# using either '==' or the regular expression
|
|
# engine. Since it is a string field, we will need
|
|
# to run it through the unencodeString function to
|
|
# change any special characters back to their
|
|
# original values.
|
|
if self.field_types[fieldPos] == str:
|
|
try:
|
|
if useRegExp:
|
|
if not pattern.search(
|
|
self._unencodeString(record[fieldPos])
|
|
):
|
|
is_no_match = True
|
|
break
|
|
else:
|
|
if record[fieldPos] != pattern:
|
|
is_no_match = True
|
|
break
|
|
except Exception:
|
|
raise KBError(
|
|
'Invalid match expression for %s'
|
|
% self.field_names[fieldPos])
|
|
# If the field type is boolean, then I will simply
|
|
# do an equality comparison. See comments above
|
|
# about why I am actually doing a string compare
|
|
# here rather than a boolean compare.
|
|
elif self.field_types[fieldPos] == bool:
|
|
if record[fieldPos] != pattern:
|
|
is_no_match = True
|
|
break
|
|
# If it is not a string or a boolean, then it must
|
|
# be a number or a date.
|
|
else:
|
|
# Convert the table's field value, which is a
|
|
# string, back into it's native type so that
|
|
# we can do the comparison.
|
|
if record[fieldPos] == '':
|
|
tableValue = None
|
|
elif self.field_types[fieldPos] == int:
|
|
tableValue = int(record[fieldPos])
|
|
elif self.field_types[fieldPos] == float:
|
|
tableValue = float(record[fieldPos])
|
|
# I don't convert datetime values from strings
|
|
# back into their native types because it is
|
|
# faster to just leave them as strings and
|
|
# convert the comparison value that the user
|
|
# supplied into a string. Comparing the two
|
|
# strings works out the same as comparing two
|
|
# datetime values anyway.
|
|
elif self.field_types[fieldPos] in (
|
|
datetime.date, datetime.datetime):
|
|
tableValue = record[fieldPos]
|
|
else:
|
|
# If it falls through to here, then,
|
|
# somehow, a bad field type got put into
|
|
# the table and we show an error.
|
|
raise KBError('Invalid field type for %s'
|
|
% self.field_names[fieldPos])
|
|
# Now we do the actual comparison. I used to
|
|
# just do an eval against the pattern string
|
|
# here, but I found that eval's are VERY slow.
|
|
# So, now I determine what type of comparison
|
|
# they are trying to do and I do it directly.
|
|
# This sped up queries by 40%.
|
|
if not pattern[0](tableValue, pattern[1]):
|
|
is_no_match = True
|
|
break
|
|
# If a 'No Match' exception was raised, then go to the
|
|
# next record, otherwise, add it to the list of matches.
|
|
if not is_no_match:
|
|
match_list.append([line, fpos])
|
|
# Save the file position BEFORE we read the next record,
|
|
# because after a read it is pointing at the END of the
|
|
# current record, which, of course, is also the BEGINNING
|
|
# of the next record. That's why we have to save the
|
|
# position BEFORE we read the next record.
|
|
fpos = fptr.tell()
|
|
line = fptr.readline()
|
|
|
|
# After searching, return the list of matched records.
|
|
return match_list
|
|
|
|
#----------------------------------------------------------------------
|
|
# _getMatchByRecno
|
|
#----------------------------------------------------------------------
|
|
def _getMatchByRecno(self, fptr, recnos):
|
|
"""Search by recnos. recnos is a list, containing '*', an integer, or
|
|
a list or tuple of integers"""
|
|
|
|
# Initialize table location marker and read in first record
|
|
# of table.
|
|
fpos = fptr.tell()
|
|
line = fptr.readline()
|
|
|
|
if recnos == ['*']:
|
|
# take all the non blank lines
|
|
while line:
|
|
# Strip of newline character.
|
|
line = line[0:-1]
|
|
|
|
if line.strip():
|
|
yield [line,fpos]
|
|
fpos = fptr.tell()
|
|
line = fptr.readline()
|
|
else:
|
|
# select the records with record number in recnos
|
|
if isinstance(recnos[0],(tuple,list)):
|
|
# must make it a list, to be able to remove items
|
|
recnos = list(recnos[0])
|
|
while line:
|
|
# Strip of newline character.
|
|
line = line[0:-1]
|
|
|
|
# If line is not blank, split it up into fields.
|
|
if line.strip():
|
|
record = line.split("|")
|
|
# If record number for current record equals record number
|
|
# we are searching for, add it to match list
|
|
if int(record[0]) in recnos:
|
|
yield [line, fpos]
|
|
recnos.remove(int(record[0]))
|
|
# if no more recno to search, stop looping
|
|
if not recnos: break
|
|
|
|
# update the table location marker
|
|
# and read the next record.
|
|
fpos = fptr.tell()
|
|
line = fptr.readline()
|
|
|
|
# Stop iteration
|
|
return
|
|
|
|
#----------------------------------------------------------------------
|
|
# _incrRecnoCounter
|
|
#----------------------------------------------------------------------
|
|
def _incrRecnoCounter(self, fptr):
|
|
# Save where we are in the table.
|
|
last_pos = fptr.tell()
|
|
|
|
# Go to header record and grab header fields.
|
|
fptr.seek(0)
|
|
line = fptr.readline()
|
|
header_rec = line[0:-1].split('|')
|
|
|
|
# Increment the recno counter.
|
|
self.last_recno += 1
|
|
header_rec[0] = "%06d" %(self.last_recno)
|
|
|
|
# Write the header record back to the file. Run each field through
|
|
# encoder to handle special characters.
|
|
self._writeRecord(fptr, 0, '|'.join(header_rec))
|
|
|
|
# Go back to where you were in the table.
|
|
fptr.seek(last_pos)
|
|
# Return the newly incremented recno counter.
|
|
return self.last_recno
|
|
|
|
#----------------------------------------------------------------------
|
|
# _incrDeleteCounter
|
|
#----------------------------------------------------------------------
|
|
def _incrDeleteCounter(self, fptr):
|
|
# Save where we are in the table.
|
|
last_pos = fptr.tell()
|
|
|
|
# Go to header record and grab header fields.
|
|
fptr.seek(0)
|
|
line = fptr.readline()
|
|
header_rec = line[0:-1].split('|')
|
|
|
|
# Increment the delete counter.
|
|
self.del_counter += 1
|
|
header_rec[1] = "%06d" %(self.del_counter)
|
|
|
|
# Write the header record back to the file.
|
|
self._writeRecord(fptr, 0, '|'.join(header_rec))
|
|
|
|
# Go back to where you were in the table.
|
|
fptr.seek(last_pos)
|
|
|
|
#----------------------------------------------------------------------
|
|
# _deleteRecord
|
|
#----------------------------------------------------------------------
|
|
def _deleteRecord(self, fptr, pos, record):
|
|
# Move to record position in table.
|
|
fptr.seek(pos)
|
|
|
|
# Overwrite record with all spaces.
|
|
self._writeRecord(fptr, pos, " " * len(record))
|
|
|
|
#----------------------------------------------------------------------
|
|
# _writeRecord
|
|
#----------------------------------------------------------------------
|
|
def _writeRecord(self, fptr, pos, record):
|
|
try:
|
|
# If record is to be appended, go to end of table and write
|
|
# record, adding newline character.
|
|
if pos == 'end':
|
|
fptr.seek(0, 2)
|
|
fptr.write(record + '\n')
|
|
else:
|
|
# Otherwise, move to record position in table and write
|
|
# record.
|
|
fptr.seek(pos)
|
|
fptr.write(record)
|
|
except:
|
|
raise KBError('Could not write record to: ' + fptr.name)
|
|
|
|
#----------------------------------------------------------------------
|
|
# _openTable
|
|
#----------------------------------------------------------------------
|
|
def _openTable(self, name, access):
|
|
try:
|
|
# Open physical file holding table.
|
|
fptr = open(name, access)
|
|
except:
|
|
raise KBError('Could not open table: ' + name)
|
|
# Return handle to physical file.
|
|
return fptr
|
|
|
|
#----------------------------------------------------------------------
|
|
# _closeTable
|
|
#----------------------------------------------------------------------
|
|
def _closeTable(self, fptr):
|
|
try:
|
|
# Close the file containing the table.
|
|
fptr.close()
|
|
except:
|
|
raise KBError('Could not close table: ' + fptr.name)
|
|
|
|
#----------------------------------------------------------------------
|
|
# _sendSocket
|
|
#----------------------------------------------------------------------
|
|
def _sendSocket(self, command):
|
|
dbSock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
|
dbSock.connect((self.host, self.port))
|
|
self.dbSock = dbSock
|
|
|
|
# Send the length of the command followed by the command itself.
|
|
dbSock.send('%16s%s' % (len(command), command))
|
|
|
|
# Receive the return value of the method.
|
|
data = ''
|
|
while len(data) < 16:
|
|
data = data + dbSock.recv(1024)
|
|
recv_length = int(data[:16])
|
|
data = data[16:]
|
|
|
|
while len(data) < recv_length:
|
|
data = data + dbSock.recv(1024)
|
|
|
|
# Convert pickled binary data back into it's original format
|
|
# (usually a list).
|
|
data = cPickle.loads(data[:recv_length])
|
|
|
|
# If the server passed back an error object, re-raise that error
|
|
# here on the client side, otherwise, just return the data to the
|
|
# caller.
|
|
if isinstance(data, Exception):
|
|
raise data
|
|
else:
|
|
return data
|
|
|
|
|
|
#--------------------------------------------------------------------------
|
|
# Generic class for records
|
|
#--------------------------------------------------------------------------
|
|
class Record(object):
|
|
"""Generic class for record objects.
|
|
|
|
Public Methods:
|
|
__init__ - Create an instance of Record.
|
|
"""
|
|
#----------------------------------------------------------------------
|
|
# init
|
|
#----------------------------------------------------------------------
|
|
def __init__(self,names,values):
|
|
self.__dict__ = dict(zip(names, values))
|
|
|
|
|
|
#--------------------------------------------------------------------------
|
|
# KBError Class
|
|
#--------------------------------------------------------------------------
|
|
class KBError(Exception):
|
|
"""Exception class for Database Management System.
|
|
|
|
Public Methods:
|
|
__init__ - Create an instance of exception.
|
|
"""
|
|
#----------------------------------------------------------------------
|
|
# init
|
|
#----------------------------------------------------------------------
|
|
def __init__(self, value):
|
|
self.value = value
|
|
|
|
def __str__(self):
|
|
return `self.value`
|
|
|
|
# I overrode repr so I could pass error objects from the server to the
|
|
# client across the network.
|
|
def __repr__(self):
|
|
format = """KBError("%s")"""
|
|
return format % (self.value)
|