Commit Graph

121 Commits

Author SHA1 Message Date
Paul Sokolovsky
6c3db26ab7 py/stream: Fix signed comparison issue. 2016-04-10 13:31:52 +03:00
Paul Sokolovsky
0c97e4c414 py/stream: Add Python-level ioctl() method.
Will call underlying C virtual methods of stream interface. This isn't
intended to be added to every stream object (it's not in CPython), but
is convenient way to expose extra operation on Python side without
adding bunch of Python-level methods.
2016-04-10 12:45:46 +03:00
Paul Sokolovsky
53ad5edc01 py/stream: Fix stupid thinko with variable naming/shadowing. 2016-03-27 12:58:33 +03:00
Paul Sokolovsky
4a02a8f74d py/stream: Fix object vs ptr usecase in mp_stream_writeall(). 2016-03-24 19:43:08 +02:00
Paul Sokolovsky
d4c8e626f2 py/stream: Add mp_stream_writeall() helper function.
Spools entire output buffer to a blocking stream (chunk by chunk if
needed).
2016-03-24 19:09:00 +02:00
Damien George
4b72b3a133 py: Change type signature of builtin funs that take variable or kw args.
With this patch the n_args parameter is changed type from mp_uint_t to
size_t.
2016-01-11 00:49:27 +00:00
Damien George
e84325bd1d py: Add mp_get_stream_raise to factor out check for stream methods. 2015-12-09 18:47:43 +00:00
Damien George
999cedb90f py: Wrap all obj-ptr conversions in MP_OBJ_TO_PTR/MP_OBJ_FROM_PTR.
This allows the mp_obj_t type to be configured to something other than a
pointer-sized primitive type.

This patch also includes additional changes to allow the code to compile
when sizeof(mp_uint_t) != sizeof(void*), such as using size_t instead of
mp_uint_t, and various casts.
2015-11-29 14:25:35 +00:00
Damien George
4e7107a572 py: Change mp_print_strn_t func type to use size_t for the str length. 2015-11-29 14:25:04 +00:00
Paul Sokolovsky
7799410950 py/stream: Allow to reuse is_nonblocking_error(). 2015-10-18 15:39:33 +03:00
blmorris
bdd78c31b6 py: Add stream_tell method, and use for unix and stmhal file tell. 2015-08-13 22:56:32 +01:00
Damien George
c50772d19f py: Add mp_obj_get_int_truncated and use it where appropriate.
mp_obj_get_int_truncated will raise a TypeError if the argument is not
an integral type.  Use mp_obj_int_get_truncated only when you know the
argument is a small or big int.
2015-05-12 23:05:53 +01:00
Damien George
0d3cb6726d py: Change vstr so that it doesn't null terminate buffer by default.
This cleans up vstr so that it's a pure "variable buffer", and the user
can decide whether they need to add a terminating null byte.  In most
places where vstr is used, the vstr did not need to be null terminated
and so this patch saves code size, a tiny bit of RAM, and makes vstr
usage more efficient.  When null termination is needed it must be
done explicitly using vstr_null_terminate.
2015-01-28 23:43:01 +00:00
Paul Sokolovsky
ca3dbb8d8b stream: readall(): Make sure there's a trailing NUL char. 2015-01-24 00:22:47 +02:00
Paul Sokolovsky
425f952a1e stream: Fix readall() implementation in respect to NUL terminator bytes.
After vstr refactor. Fixes #1084.
2015-01-23 17:59:37 +02:00
Damien George
05005f679e py: Remove mp_obj_str_builder and use vstr instead.
With this patch str/bytes construction is streamlined.  Always use a
vstr to build a str/bytes object.  If the size is known beforehand then
use vstr_init_len to allocate only required memory.  Otherwise use
vstr_init and the vstr will grow as needed.  Then use
mp_obj_new_str_from_vstr to create a str/bytes object using the vstr
memory.

Saves code ROM: 68 bytes on stmhal, 108 bytes on bare-arm, and 336 bytes
on unix x64.
2015-01-21 23:18:02 +00:00
Damien George
0b9ee86133 py: Add mp_obj_new_str_from_vstr, and use it where relevant.
This patch allows to reuse vstr memory when creating str/bytes object.
This improves memory usage.

Also saves code ROM: 128 bytes on stmhal, 92 bytes on bare-arm, and 88
bytes on unix x64.
2015-01-21 23:17:27 +00:00
Damien George
51dfcb4bb7 py: Move to guarded includes, everywhere in py/ core.
Addresses issue #1022.
2015-01-01 20:32:09 +00:00
Damien George
be6d8be91e py: Rename mp_obj_int_get to mp_obj_int_get_truncated; fix struct.pack.
mp_obj_int_get_truncated is used as a "fast path" int accessor that
doesn't check for overflow and returns the int truncated to the machine
word size, ie mp_int_t.

Use mp_obj_int_get_truncated to fix struct.pack when packing maximum word
sized values.

Addresses issues #779 and #998.
2014-12-05 23:13:52 +00:00
Damien George
5694cc5490 py: Make stream seek correctly check for ioctl fn; add seek for textio. 2014-11-16 23:56:37 +00:00
Paul Sokolovsky
838eb1fa2d stream: Implement seek operation support via ioctl, wrapped in generic method.
Also, implement for unix port.
2014-11-17 00:16:14 +02:00
Damien George
d00d8ac95c py: Use mp_uint_t where appropriate in stream functions. 2014-10-24 11:26:12 +00:00
Paul Sokolovsky
e2f8d98525 stream: Add optional 2nd "length" arg to .readinto() - extension to CPython.
While extension to file.readinto() definition of CPython, the additional arg
is similar to what in CPython available in socket.recv_into().
2014-10-23 21:43:59 +03:00
Paul Sokolovsky
1a55b6a787 unix, stmhal: Implement file.readinto() method.
Also, usocket.readinto(). Known issue is that .readinto() should be available
only for binary files, but micropython uses single method table for both
binary and text files.
2014-10-18 22:44:07 +03:00
Damien George
b7a4b0f86f py: Improve stream_read so it doesn't need to alloc 2 bits of heap. 2014-10-17 23:34:06 +01:00
Paul Sokolovsky
923a8a8320 stream: Handle non-blocking errors in readline() properly.
Just like they handled in other read*(). Note that behavior of readline()
in case there's no data when it's called is underspecified in Python lib
spec, implemented to behave as read() - return None.
2014-10-16 12:22:52 +03:00
Paul Sokolovsky
0c7b26c0f8 stream: Return errno value as first arg of OSError exception.
This is CPython-compatible convention established yet in acb13886fc.
2014-10-16 02:58:52 +03:00
Damien George
69b7dae362 py: Small cleanup in stream.c. 2014-08-22 18:30:02 +01:00
Damien George
adf0f2ae1a py: Change stream protocol API: fns return uint; is_text for text. 2014-07-27 22:38:58 +01:00
Paul Sokolovsky
4ecb700fe3 streams: Treat non-error output size as unsigned. 2014-07-23 00:25:46 +03:00
Paul Sokolovsky
e1b1abc1e8 stream: Revert to checking for the correct error value. 2014-07-23 00:23:44 +03:00
Dave Hylands
1d8816c36b Deal with reading a buffer less than what was allocated.
With this fix, file_long_read now passes.
2014-07-21 19:10:10 -07:00
Damien George
1694bc733d py: Add stream reading of n unicode chars; unicode support by default.
With unicode enabled, this patch allows reading a fixed number of
characters from text-mode streams; eg file.read(5) will read 5 unicode
chars, which can made of more than 5 bytes.

For an ASCII stream (ie no chars > 127) it only needs to do 1 read.  If
there are lots of non-ASCII chars in a stream, then it needs multiple
reads of the underlying object.

Adds a new test for this case.  Enables unicode support by default on
unix and stmhal ports.
2014-07-19 18:34:04 +01:00
Paul Sokolovsky
ac736f15c9 stream: Factor out mp_stream_write() method to write a memstring to stream. 2014-07-13 23:14:32 +03:00
Damien George
40f3c02682 Rename machine_(u)int_t to mp_(u)int_t.
See discussion in issue #50.
2014-07-03 13:25:24 +01:00
stijn
ec6fa8732b windows: Sync mpconfigport.h with the unix' version
- rearrange/add definitions that were not there so it's easier to compare both
- use MICROPY_PY_SYS_PLATFORM in main.c since it's available anyway
- define EWOULDBLOCK, it is missing from ingw32
2014-06-29 09:40:20 +02:00
Paul Sokolovsky
f5f6c3b792 streams: Reading by char count from unicode text streams is not implemented. 2014-06-27 00:04:20 +03:00
Paul Sokolovsky
e22cddbe2a stream: Use mp_obj_is_true() for EOF testing.
Getting a length of string may be expensive, depending on the underlying
implementation.
2014-06-13 23:53:10 +03:00
Damien George
f600a6a085 py: Slightly improve efficiency of mp_obj_new_str; rename str_new.
Reorder interning logic in mp_obj_new_str, to be more efficient.

str_new is globally accessible, so should be prefixed with mp_obj_.
2014-05-25 22:34:34 +01:00
Paul Sokolovsky
a47b64ae2d objstringio: Implement io.BytesIO.
Done in generalized manner, allowing any stream class to be specified as
working with bytes.
2014-05-15 07:28:19 +03:00
Paul Sokolovsky
da9f0924ef py, unix: Add copyright for modules I worked closely on. 2014-05-13 18:41:25 +03:00
Paul Sokolovsky
0ef015b253 stream: Make non-blcoking stream support configurable.
Enable only on unix. To avoid unpleasant surprises with error codes.
2014-05-07 02:25:45 +03:00
Paul Sokolovsky
b9be45e421 stream: Use standard name of DEFAULT_BUFFER_SIZE. 2014-05-07 02:17:14 +03:00
Paul Sokolovsky
6e73143de8 stream: Add compliant handling of non-blocking readall(). 2014-05-07 02:17:14 +03:00
Paul Sokolovsky
a592104acd stream: Add compliant handling of non-blocking read()/write().
In case of empty non-blocking read()/write(), both return None. read()
cannot return 0, as that means EOF, so returns another value, and then
write() just follows. This is still pretty unexpected, and typical
"if not len:" check would treat this as EOF. Well, non-blocking files
require special handling!

This also kind of makes it depending on POSIX, but well, anything else
should emulate POSIX anyway ;-).
2014-05-07 02:17:14 +03:00
Damien George
04b9147e15 Add license header to (almost) all files.
Blanket wide to all .c and .h files.  Some files originating from ST are
difficult to deal with (license wise) so it was left out of those.

Also merged modpyb.h, modos.h, modstm.h and modtime.h in stmhal/.
2014-05-03 23:27:38 +01:00
Damien George
d5f5b2f766 py, stream: Implement readlines for a stream. 2014-05-03 22:01:32 +01:00
Paul Sokolovsky
f54bcbf099 py, unix: Make "mpconfig.h" be first included, as other headers depend on it.
Specifically, nlr.h does.
2014-05-02 17:48:40 +03:00
Paul Sokolovsky
45fb143ba6 streams: Make .write() support arbitrary objects with buffer interface.
This in particular fixes writing str vs bytes.
2014-04-26 05:54:16 +03:00
Damien George
ea8d06c39d py: Add MP_OBJ_STOP_ITERATION and make good use of it.
Also make consistent use of MP_OBJ_NOT_SUPPORTED and MP_OBJ_NULL.
This helps a lot in debugging and understanding of function API.
2014-04-17 23:19:36 +01:00
Ilya Dmitrichenko
5630b01920 py: the entire <unistd.h> shouldn't be needed 2014-04-12 16:45:35 +01:00
Damien George
27e735fd18 py: Replace stream_p with *stream_p in mp_obj_type_t.
This is to reduce ROM usage.  stream_p is used in file and socket types
only (at the moment), so seems a good idea to make the protocol
functions a pointer instead of the actual structure.

It saves 308 bytes of ROM in the stmhal/ port, 928 in unix/.
2014-04-05 23:02:23 +01:00
Damien George
ea13f407a3 py: Change nlr_jump to nlr_raise, to aid in debugging.
This does not affect code size or performance when debugging turned off.

To address issue #420.
2014-04-05 18:32:08 +01:00
Damien George
66eaf84b8c py: Replace mp_const_stop_iteration object with MP_OBJ_NULL. 2014-03-26 19:27:58 +00:00
xbe
efe3422394 py: Clean up includes.
Remove unnecessary includes. Add includes that improve portability.
2014-03-17 02:43:40 -07:00
Damien George
c5966128c7 Implement proper exception type hierarchy.
Each built-in exception is now a type, with base type BaseException.
C exceptions are created by passing a pointer to the exception type to
make an instance of.  When raising an exception from the VM, an
instance is created automatically if an exception type is raised (as
opposed to an exception instance).

Exception matching (RT_BINARY_OP_EXCEPTION_MATCH) is now proper.

Handling of parse error changed to match new exceptions.

mp_const_type renamed to mp_type_type for consistency.
2014-02-15 16:10:44 +00:00
Paul Sokolovsky
520e2f58a5 Replace global "static" -> "STATIC", to allow "analysis builds". Part 2. 2014-02-12 18:31:30 +02:00
Damien George
698ec21e46 Make mp_obj_str_get_data return char* instead of byte*.
Can't decide which is better for string type, char or byte pointer.
Changing to char removes a few casts.  Really need to do proper unicode.
2014-02-08 18:17:23 +00:00
Paul Sokolovsky
0914371faf file.readline(): Use mp_obj_str_get_data() and fix off-by-one error on EOF. 2014-01-22 19:17:58 +02:00
Damien George
5fa93b6755 Second stage of qstr revamp: uPy str object can be qstr or not. 2014-01-22 14:35:10 +00:00
Damien George
12eaccacda Merge branch 'master' of github.com:micropython/micropython
Conflicts:
	py/objstr.c
	py/py.mk
	py/stream.c
	unix/main.c
	unix/socket.c
2014-01-21 21:54:15 +00:00
Damien George
55baff4c9b Revamp qstrs: they now include length and hash.
Can now have null bytes in strings.  Can define ROM qstrs per port using
qstrdefsport.h
2014-01-21 21:40:13 +00:00
Paul Sokolovsky
ff3bdea49d stream_read(): Shrink memory block to actual read size. 2014-01-20 20:37:01 +02:00
Paul Sokolovsky
d54bef7692 stream: Add generic unbuffered iternext method.
Uses stream_unbuffered_readline underline.
2014-01-20 18:42:08 +02:00
Damien George
a11ceca807 Change int to uint for n_args in function with variable arguments. 2014-01-19 16:02:09 +00:00
Paul Sokolovsky
a671f891dd Make file.read() and file.read(-1) call out to file.readall().
Per Python3 io module semantics.
2014-01-16 12:58:36 +02:00
Paul Sokolovsky
323c09e881 Do not assume that vstr buf is the same after it was extended. 2014-01-16 12:58:18 +02:00
Paul Sokolovsky
9953ca432b Add unbuffered readline() implementation for Raw I/O files. 2014-01-15 23:43:25 +02:00
Paul Sokolovsky
5225450b9f Add generic impl of stream .readall() method. Use one for unix io.FileIO. 2014-01-13 23:31:06 +02:00
Damien George
6c73ca1e75 py: add variable argument exception constructor function.
Addresses issue #104.
2014-01-08 18:11:23 +00:00
Paul Sokolovsky
e98cf40c34 Add generic implementations of Python read()/write methods for streams.
These can be used for any object which implements stream protocol
(mp_stream_p_t).
2014-01-08 17:38:38 +02:00