Hello, and welcome to Chapter Eight: Python Lists.
So now we're sort of going to start taking care of business.
We are doing, make lists and
dictionaries and tuples and really start manipulating this data,
and doing real data analysis, starting the,
laying the proper work for real data analysis.
As always, these lectures, audio, video, slides,
and even book are copyright Creative Commons Attribution.
So, lists, dictionaries, and tuples, the next real three big topics we're going to
talk about, are collections. And we've been doing lists already, right?
We've been doing lists when we were doing for loops.
A list in Python is something that has a square braces.
This is a constant list.
Now, when I first talked to you
about variables, I sort of oversimplified things.
I said
if you put like x equals two, and then put
x equals four, the two and the four overwrite each other.
A collection is where you can put a bunch of things in the same variable.
Now, I have to have a way to find those things.
But it allows us to put multiple things in
more, more things, more than one thing in the variable.
So, here we have friends, that has three strings, Joseph, Glenn, and Sally.
And we have carryon
that has socks, shirt, and perfume. So that's the basic idea.
So what's not a collection?
Well, simple variables.
Simple variables are not collections, just like this example.
I say x equals 2, x equals 4, and print x,
and the 4's in there and the 2 is somehow gone.
It was there for a moment, and then it's gone.
And so that's a normal variable.
They're not collections. You can't put more than one thing in it.
But when you put more than one thing in it, then you
have to have a way to find the things that are in there.
We'll, we'll get to that.
So, we've been using list constants for the last couple
of chapters just because we have to use list constants.
You know, so we used, in the for loop chapter, we did lists of numbers.
We have done lists of strings, that's strings, red, yellow, and blue.
And you don't have to necessarily, you don't necessarily
have to have things all of the same type.
This is a three-item list, that has a string red,
the number integer 24, and 98.6, which is a floating point number.
And here's an interesting thing, just as a side note.
This shows that floating point numbers are
not always perfectly represented inside of the computer.
It's sort of an artifact of how they work.
And this is an example of 98.6 is really 98 point
na, na, na, na, na.
So, but, don't, when you see something like that, don't freak out.
Floating point numbers are the ones that show this behavior.
So, interestingly, you can always, although we won't put a lot of energy into
this, you can also have an element of a list be a list itself.
So this a outer list that's got three elements.
1, 7, and then
a list that's 5 and 6.
So, if you look at the length of this, there is three things in it.
Not four, three.
Because the outer list has 1, 2, 3 things in it.
And an empty list is bracket, bracket.
Okay?
Like I said, we have been going through lists all along.
We have iteration variables for i in.
This is a list. We've been using it all along.
Similarly, we've been using lists in definite loops, are a
great way to go through lists, for friend in friends, there we have
goes through three times, out come three lines, with the
variable friend advancing through the three successive items in the list.
And away we go.
So, again, lists are not completely foreign to us.
Now,
just like in a string, we can use the index operator,
the square bracket operator, and we can look up items in the list.
Sub one, friends, sub one.
Not surprisingly, using the European elevator rule,
the first item in a list is sub zero, the second
item is sub one and the third one is sub two.
So here when I print friends sub one I get Glenn.
Which is the second element. Just like strings.
So once you kind of know it for strings, lists
and the rest of these things make a lot more sense.
Just, remember that we're in Europe, and things start with zero.
Some things in these data items that we work with are not mutable.
So for example, strings, when we ask for a lower case
version of a string, we're given a copy of that string.
And that's because strings are not mutable, and we can see this
by doing something like saying fruit sub 0 equals lowercase b.
Now you'd think that that would just change this
to be a lower case b, but it doesn't, okay?
It says string object does not support item assignment
which means that you're not allowed to reassign.
You can make a new string and put different things in
that new string, but once the strings are made, they're not changeable.
And that's why when we call fruit.lower, we get a copy of it in lower case.
And so x is a copy of the original string, but
the original string, once we assign it into fruit, is unchanged.
It can't be changed.
Lists, on the other hand, can be changed, and we
can change them in the middle.
This is one of the things we like about them.
So here we have a list: 2, 14, 26, 41, and 63.
Then we say lotto sub two.
Of course, that's going to be the third item.
Lotto sub two is equal to 28.
Then we print it and we see the new number there.
So all this is saying is that we can change them, right?
Strings no, and lists yes.
You can change lists, but you can't change strings.
So the len function, we've used it for several
things, we can say you know, use, len is
used for, for strings and it's used for lists as well.
So the same function knows when its
parameter is a string. And when its parameter is a string,
it gives us the number of characters in the string.
And when it is a list, it gives us
the number of elements in the list.
And just because one of them is a string, it's still one element from the point
of view of this list.
So it has one, two, three, four - four items in the list, okay?
So, the range function is a special function.
It's probably about time to talk about the range function.
The range function is a function that generates a list, that
produces a list and gives it back to us.
And so you give the range function a
parameter, how many items you want, and the range
function creates and gives us back a list that
is four numbers starting at zero, which is zero
up to, but not including three. Sound familiar?
Yeah.
Zero up to but not, I mean zero up to, but not including four.
And, and so the same thing is true here. So, we can combine the len and the range
to say, you know, to say okay, well len friends, that's three
items, and range len friends is 0, 1, 2. And it also
corresponds exactly to these items. So we can actually use this
to construct loops to go through a list. We already have a basic for loop, right?
We basically have a for loop that is our,
that, that said that for each friend in friends.
And out comes, Happy New Year, Glenn and Joseph.
If we also want to know where, what position we're at as
the loop progresses, we can rewrite the exact same loop a different way.
And make i be our iteration variable.
And say i in range(len(friends)), that turns this into zero, one, two.
And then i goes zero, one, two.
And then, we can in the loop, look up the
particular friend that is the particular one we are interested in,
using the index operator, friend sub i.
And then print Happy New Year.
So these two loops,
these two loops are equivalent. These, oop, not that one.
[SOUND] This loop and this loop. This loop is
preferred, unless you happen to need this value i, which tells you where you're at.
In case maybe you're going to change something, you're
going to look through something and then change it.
So, but, but, for what I've written here, they're exactly equivalent.
Prefer the simpler one, unless you need
the more complex one. They both produce the same kind of output.
We can concatenate lists, much like we concatenate strings, with plus.
And you can think of the Python operator's looking to its right and to its left and
saying oh, those are both lists, I know what
to do with lists, I'm going to put those together.
And so that produces a two, three-long lists become a six-long
list with the first one followed by the second one concatenated.
It didn't hurt the original, a. c is a new list, basically.
We can also slice lists. Feels a lot like strings, right?
Everything's kind of like strings.
For loops like strings, concatenation like strings, and now slicing like strings.
And it is exactly the same.
So one up to, but not including. Just remember, up to, but not including.
the second parameter, is up to but not including, so that starts at the sub one,
which is the second one up to but not including 3, the third one, so.
This is 1, 2, and 3 so that's 41 comma 2.
Starting at the first one, up to but not including the third one.
We can similarly eliminate the first one,
so that's up to but not including the fourth one.
Starting at zero, one, two, three, but not including four.
So that's this one. If we go three to the end, and again,
remember that there, starting at 0, so 3 to the end is 0, 1, 2, 3 to the end.
The number 3 doesn't matter. So that's 3, 74, 15.
And the
whole thing, that's the whole thing, so these two things are the same.
So slicing works like strings, starting and up
to but not including is the second parameter.
There are some methods, and you can
read about these online in the Python documentation.
We can use the built-in function.
It doesn't have a lot of use in sort of how
we run, when we're running programs but it's kind of of useful.
I like it when I'm typing
interactively. Like, what can this thing do?
So I make a list, list is a unique type, and
I say, with dir I say what can we do with it?
Well, we can append, we can count, extend, index, insert, pop, remove, reverse
and sort. And then you can sort of read up on all these things.
I'll show you just a couple. We can build a list with the append.
So this syntax here,
stuff equals list, that's called a constructor
which says give me an empty list.
You could also say bracket, bracket for an empty list.
Whatever, you gotta make an empty list and then you call the append.
Remember that lists are mutable, so it's okay to change it.
So we're saying, okay, we started with an empty list.
Now append to the end of that, the word book.
And then append to that, 99.
Wait a sec.
That's a mistake.
So I have to fix this mistake.
So watch me fix the mistake. Poof.
Now my thing is magically fixed. Isn't that amazing.
I have magic powers when it comes to slide fixing.
I just snap my fingers and the slides are fixed.
So here we go.
We append the 99, and we print it out.
And it's got book and 99, emphasizing the fact that they don't
have to be the exact same kind of thing in a list.
Then later we append cookie and then it's book, 99, cookie.
Okay? So this append, we won't do it in line
like this so often, we'll tend to do it in a loop as we're building up a
list, but that's the way you start with
an empty list and then [SOUND] programmatically grow it.
We can ask, much like we do in a string, we can ask if an item is in a list.
So here is a list called some, with these numbers in it.
It's got five numbers in it.
Is nine in some? True, yes it is.
Is 15 in some? False.
Is 20 not in, that's a leg, a legal syntax, that is legal syntax.
Is 20 not in some, yes it's not there, okay?
They don't modify the list, don't modify the list, they're just asking questions.
These are logical operations often used in if statements or
while, some kind of a logic that you might be building.
Okay, so lists have order.
So when we were appending them, the first thing went
in first, the second thing went in second, et cetera, et cetera.
And we can also tell the list to sort itself.
So one of the things that we can do with a list,
now we're starting to see some power here, is say, sort yourself.
This is a list of strings.
It can sort numbers, it can sort lots of things.
friends.sort, that says hey there, dear friends, sort yourself.
This makes a change.
It alters the list, and puts it, in
this case, in alphabetical order, Glenn, Joseph, and Sally.
It is muted, it was, it's, it's been modified, and so
friend sub one is now Joseph because that's the second one.
Okay?
So the sort method says sort yourself now,
sort yourself, and it sorts and then it stays sorted.
So [COUGH]
you're going to be kind of ticked about this particular slide.
Because there's a whole bunch of built-in functions that help with lists.
And, there's max, there's min, there's len, various things.
And so we could, all those loops that I told you how to
do, I was just showing you that stuff because I thought it was important.
This the simplest way to go through and
find the largest, smallest, and sum, et cetera.
So here's a list of numbers.
We can say how many are there.
That's the count. We can say what's the largest, it's 74.
What's the smallest, that'd be 3.
What is the sum of the running total of them all? 154.
If you remember from a few lectures ago, these are the same numbers.
And what is the average, which is, sum of them over the length of them,
Okay?
So this makes a lot more sense and if you had a list of numbers
like this, you would simply say what's the max, you wouldn't write a max loop.
I just did that to kind of demonstrate how loops work.
[COUGH] Demonstrate how loops work.
So here is a way that you can sort
of change those kind of programs that we wrote.
So there's two ways to write a summing program.
Let's just say instead of the data being
in a list, we're going to write a while loop that's going to read a
set of numbers until we say done, and then compute the average of those numbers.
Okay, so let's say this is our problem.
Read a list of numbers, wait till the word done comes in, and then average them.
So here's a little program that does that.
We create total equals zero, count equals zero.
Make a infinite loop with while True.
And then we ask
to enter a number.
We get a string back from this, remember raw_input always
gives us strings back, and then if it's done, we're going to break.
This is the version of the if that does not require an indent.
We just put the break up there.
And so that gets us out of the loop when the time is right.
So when the time is right over here.
And then, we convert the value to float.
We use a float to convert the input to a floating point number.
And then we do our accumulation pattern,
total equals total plus value, count equals count plus one.
So this is going to run.
These numbers are going to go up and up and up and up.
And then we're going to break out of it,
calculate the average, and then print the average.
Because that's a floating point number, so now the average is a floating point number.
So that's one way to do it.
Right?
That would be one way to write a program
that does an average, is keep a running average
as you're reading the numbers.
But there's another way to do it, that would exact, work exactly
the same way, and this is when you can start using lists.
So you come in, you say I'm going to make a list
of numbers, just a mnemonic name, numlist, is an empty list.
Then I create another infinite loop that's going to read for enter a number.
And if it's done, break.
That gets us out of it. Convert the value to an int.
Convert the value to a float, the input value to a float.
And then append it to the list.
So now the list is going to grow, each time
we read a number the list is going to grow.
However many times we add the number is
how many things are going to be in the list.
So in this case, when we're at this point and we
type done, there will be three numbers in the list, because we
will have run append three times. We'll have appended 3, 9, and 5.
We'll have them sitting in a list. And we will have exited the loop.
So now you say, oh add up all the numbers in
that list, and then divide it by the length of the list.
And print the average.
So these two programs are basically equivalent.
The only time that they might not be
equivalent was if there was ten million numbers.
This would use up 40 megabytes of your memory, which
is actually not a lot of memory on some computers.
But if memory mattered, this does store all those numbers.
This one actually just runs the calculation.
So if there's a really large number of numbers, this would make a difference,
because the list is growing and keeping them all, summing them all at the end.
This is actually storing very little data.
But for reasonably sized numbers,
like thousands or even hundreds of thousands of numbers, these
two approaches are kind of equivalent. And then sometimes you actually
want to accumulate something a little more complex than this, you want to
sort them or look for the maximum and look for something else.
Who knows what, but the notion of make a
list and then append something to the list
each time through the iteration, and then do something with
the list at the end is a rather powerful pattern.
So this is also a powerful pattern, this is accumulator
pattern where we just have the variables accumulating in the loop.
This one is one where we accumulate the data in
the loop and then do the computations all at the end.
The, certain situations will make use of these different techniques.
Okay. So, connecting strings and lists.
So there's a method, a capability
of strings that is really powerful when it comes to tearing data apart.
It's called the split. So here is a string
with three words and it has blanks in between here.
And abc.split says parse this string,
look for the blanks, break the string into pieces, and give me back a
list with one item for each of the words in the list as
defined by the spaces. Okay?
So, it takes, breaks it into three pieces and gives us that back in a list.
This is very powerful. Okay?
So we're going to split it and we get back a list.
There are three words, and the first word, stuff sub zero, is With.
So there's a lot of parsing going on here.
We could do this with for loops and a lot of other things.
There would be a lot of work in this split.
Given that this is a really common task, it's really
great that this has been put into Python for us.
Okay?
So split breaks a string into parts and produces a list of strings.
We think of these as words, we can access a
particular word or we can loop through all the words.
So here we have stuff again and now we have a, a for loop
for each of the, that's going to go through each of the three words.
And then it's going to run three times.
Now chances are good we're going to do
something different other than just print them out.
But you see how that you quickly can take a split followed by a for, and then write
a loop that's going to go through each of the
words, without working too hard to find the spaces.
You let Python do all the hard work of finding the spaces.
Okay?
So let's take a look at a couple of samples.
Just a couple of things to teach you a little more about split.
Split looks at many spaces as equal to one space.
So, if you split a lot blank, blank, blank of spaces, it's
still just throws away all the spaces and gives us four words.
One, two, three, four and throws away all the spaces,
because it assumes that's what we want done.
So that's nice.
You can also have split, you can also have split,
split on some other character. Sometimes you'll be getting data
and they'll have used a semicolon, or a comma, or
a colon, or a tab character, who knows what they've
used, and your job is to dig that data out.
So you can split, based on the different character.
Here, if we're splitting normally with, with this is a normal split.
It's not going to see the semicolons, it's looking for a space.
And so all we get back is one
item in the string, with the semicolons.
But, if we switch, and we pass semicolon
as a parameter, in as as parameter to split,
then it will know to split it based on
semicolons, and gives us first, second, and third back.
Okay?
And then it gives us three words.
So you can split either on spaces, or you
can split on a character other than a space.
Okay?
[COUGH]
So, let's take a look at how we might turn this into some of our common assignments
that we have in this chapter, where we're going to read some of the mailbox data. Okay?
So, here we go with a little program.
First three lines, we write these a lot. Open the file.
Write a for loop to loop through each
line in the file.
Then we're going to strip off the white space at the end of the line.
One, two, three. Do those all the time.
And we're looking for lines, if you look at the whole file,
we're looking for lines that start with from, followed by a space.
So if the line does not start with from
followed by a space, that's a space right there, continue.
So that's a way to skip all the lines that don't look like this.
There're thousands of lines in this file and just a few that look like this. Okay?
So we're going to look and we're going to try
to find what day of the week this thing happened on.
So, so we're throwing away all the lines with this little bit of code.
Then what we do is we take the line, which is all of this text, and then we split it.
And we know that the day of the week is words sub two.
So this is words sub zero, this is words sub one, and this is words sub two.
So this is words sub zero, sub one, and sub two.
And so, all we have to do is print out the sub two
and we get, we throw away all the lines except the from lines.
We split them and take the sec, uh, the,
the third word or words sub two and we
can quickly quickly create something that's extracting
the day of the week out of these.
Okay?
So it's, it's, I mean, it's quick, because split does the tricky work.
If you go back to the strings chapter, you saw that
we did a lot of work to get this to happen.
So here's even another tricky pattern.
So let's say we want to do what we did at the end of Chapter Six,
the string chapter.
Let's say we wanted to get back this little bit of data.
Okay?
So, can look at this and say, okay, let's split this.
And this will be zero, one, and two, and three, and four, and five, and six.
We're splitting it based on spaces.
Then the email address is words sub one, right?
So that email address is this little bit of stuff
because it's in between spaces, right? So that's what we pull out.
The email address is words sub one.
We've got that.
So that's sitting in this email address variable.
Then we really, all we want, we don't really want the whole thing,
we just want the part after the
at sign, and we can do a lookup for the, oop.
We can do a lookup of the at sign.
But you can also then do a second, come back, come back.
[SOUND] There we come.
You can also do a second split. Okay?
So we're taking this variable here, email,
which is merely this little part right here.
And we are splitting it again, except this
time we're splitting it based on a at sign.
Which means it's going to bust it right here, and find
us two pieces.
So pieces now is a list where the sub zero item is the
person's name and sub one item is the host that their mail address is held from.
Okay? And so then all we need to know is pieces
is sub one, and pieces sub one is this guy right here.
So that's pieces sub one, and so we pulled it out.
So if you go back to how we did it before, we were
doing searching, we were searching some more, and then we were taking slices.
This is a little more elegant, okay?
Because really, we split it and then we split it,
and we knew what piece we were looking at.
So this is what I call the Double Split Pattern, where you split a string
into a list, then you take a thing out, and then you split it again.
Depending on what data you're looking for.
This is just a technique, it's not the only technique.
Okay, so that's lists.
We talked about the concept of a
collection where lists have multiple things in it.
Definite loops, again, we've seen these things.
We're kind of, it looks a lot like strings
except the elements are more powerful and they're more mutable.
We still use the bracket operator and we redid the max, min, and sum.
Except we did it in, like, one line rather than a whole loop.
And something we're going to play with a lot is using split to parse strings,
the single split, and then the double split
is the natural extension of the single split.
So, see you in the next lecture, looking forward to talking about dictionaries.