CSE 390HA - 122 Honors Seminar

• the history of segregated schools and pools is relevant to computer science education (CS Ed) the 2000s
• metaphorically, being "stuck in the shallow end" happens when you don't have access to the right skills and resources. This is just like CS Ed: it's not a lack of interest, but a lack of resources.
• similar to swimming, insecurity is a huge barrier to entry for CS. These insecurities stem from historical elements of society.
• expectations are built from existing societal structures - similar to the floating line that separates the shallow and deep ends of a pool, students may feel like an invisible expectation prevents them from doing CS.
• in both cases, we often don't solve the true, structural problem - often ignoring it or applying a bandaid fix.

• the book explicitly discusses the stereotype that computer scientists are white or Asian (or Indian); this pressure feels real!
• similar identity issues around being a "math-oriented", "STEM-oriented", or "writing-oriented" person.
• computer scientists are perceived as technical, concise speakers, and good at math - even though CS is much more than that!
• a relationship between "these groups are good at math" to "these groups are good at computing"
• core underlying issue: representation. Not seeing someone who looks like you in computer science impacts if you'd engage yourself.
• as a class, we then tried to name famous swimmers (Michael Phelps, Katie Ledecky, and Ryan Lochte) - who were all white.
• as a class, we then tried to name famous computer scientists (Bill Gates, Steve Jobs, Tim Cook, Paul Allen, Ada Lovelace, Alan Turing, Richard Feynman, "the people from Hidden Figures"). Matt commented that typically, people are unable to name women, and that none of these people are asian - even though that's a heavy stereotype of software engineers.
• "Asian" isn't a monolithic categorization, and depends on what you count as Asia (e.g. is Russia in Asia? the Middle East?). Participation and inclusion is very different for East Asians, Southeast Asians, and further breakdowns.

Note from Matt: going to mostly omit the specific points here since they're personally identifying. Instead, these were just the broad-strokes themes - not a play-by-play of the conversation.

• large difference in expected graduation rates (~50% versus always 100%)
• in a different state, had single-digit Latino/a representation - "almost too few people to discriminate against".
• connection to generational expectations in the book. Some students grew up in engineering centers where everybody had at least one parent as an engineer: so, for them and their peers, tech was highly valued, and going to college was highly valued - the expectation was that you'd do it!
• stark differences for many students who moved during their childhood, with different emphasis placed on technological education, "traditional" education, and sports. [note from Matt: this was a big chunk of the conversation, but it's hard to do this without leaking info about each student!]
• in IB schools, taking computer science is awkward with the rest of the curriculum.
• in affluent areas in Washington, some schools had many, many CS classes (including cybersecurity and robotics)! Perhaps influenced by the demographics of the parents, more of which work in tech. Yet, some of our students still didn't take them!

• book mentions "technologically rich, educationally poor". The government has funded programs, but have relied on (new) organizations like Code.org to build out courses and pedagogy.
• Scratch is much more popular and is frequently used to teach kids programming!
• the field of computer science has skyrocketed (especially with social media), with a bigger emphasis on it being a great career, having prestige, and being put on a pedestal
• everything is now digital: menus, QR Codes, even Disneyland! But, access to technology has not improved proportionally, which widens the technological gap.
• not having computer skills puts a harsher limit on your socioeconomic status.
• separate from access, teachers are now more overworked, burned out, and (sometimes) hate their jobs! Giving them more technology will not solve the problem by itself - broader reform is needed.
• accessibility has two different meanings in computer science, but both are related. Connection to CSE 121 reflection TED talk on "CS Unplugged": how to teach CS on pen and paper.

• argument: Word is interacting with software, but CS is more about writing something out to accomplish a function/goal
• counterpoint: Excel does this too! You can write formulas that do most things Java methods can do (including 121 P3). Does this mean that Excel is the most used programming language in the world?
• in "Stuck in the Shallow End", there is a specific difference between computer literacy (using computers) and computer science (thinking about computers).
• everybody should learn computer literacy, it should be a mandatory part of K-12.
• but, computational thinking - and computer science - is not so different from other sciences (e.g. learning Java is like designing a lab, analyzing results - just with different tools). So, can think of this as adding another "science" (similar to biology or physics).
• computer science is not just programming - there's also an element of how computers work. For example, in AP CS Principles (AP CSP), you learn how wifi works!
• depending on how deep you go into Excel, it could be computer science? But, literacy is more about font sizes and solving "simple" problems. CSE 121's problems take hours to dive in to.
• CSE classes are about demystifying computers. Learning to use Excel is like learning to label pictures, while CS classes are like describing them in your own words and analyzing their meaning. CS is learning how these computing tools work and how you apply them in your lives.

• if we do, we need to improve access to technology (via education reform) - you need a system that guarantees access to computers for all students.
• personally regret not taking CS, even though the high school had it. Avoided it because of the association that "only smart kids take it", and if it was required, could have dispelled that notion.
• completely agree: same experience in high school, and only realized after doing 121. While this doesn't "fix" the perception/insecurity issue fully, it helps quite a bit!
• had Scratch in middle school - this felt like a good balance, sicne it wasn't a heavy requirement but still left time for exploration.
• if we add another requirement, could delay graduation more. Some students already struggle to graduate because they can't pass their required classes - this would make it worse!
• earlier exposure in middle school (with simpler content and lower stakes) may be helpful.
• also, many bad experiences with high school CS teachers - perhaps mandatory requirements induce more demand for good teachers?
• or, we'd run out of good CS teachers - and we'd have a big problem!!
• requiring CS feels idealistic - would be great if we had infinite resources, but we don't.
• if the requirement is "just have a CS class", that's too broad! Especially since school curricula are often very, very specific items with learning objectives. What would these be?
• one model: don't require it, but let it fulfill a "bundle" like math or a science (instead of physics or biology), and make sure that it hits the same learning objectives.
• going back to "Stuck in the Shallow End": students didn't take CS then because they didn't know what it was. We have the same problem now, and requiring it could help fix this!
• if CS is optional, could result in less emphasis on making sure that all students can access a computer.
• feels related to "test-optional" applications, which slightly backfired (by exacerbating existing equity gaps).

• the Xbox Adaptive Controller: did not grow up with video game consoles, and didn't think about the implications this had on disabled gamers.
• had not heard of the difference between subtitles and captions before the reading, but definitely makes sense in retrospect.
• shocked to hear that people deliberately give others seizures via flashing images, and sad that research is needed to defend against this.
• reading ability was a surprising but important one. It was jarring to see how out-of-date school standards can be.
• the adaptive controller had many different use-cases depending on the relevant disability (e.g. pressing the buttons with many different body parts), and that seems related to broader ideas when designing for accessibility.
• was familiar with the idea of switches, but had not realized that breath-controlled devices were switches (and is curious to learn more)!

• short, concise text can be very helpful to those who have ADHD or other attention deficit disabilities.
• outside of disability, it's very helpful to those who don't speak English as a first language - e.g. in helping grandparents understand documents.
• important for things like taxes and bank statements to be readable by others in simple language. But, this is different from academia, where more nomenclature might be needed.
• similar to other ADA standards (e.g. all buildings need to be built to specific codes), it would be great if all government documents, bank statements, etc. were readable.
• also similar to conversations around absurdly long terms & conditions for apps (you shouldn't need a law degree to understand the terms & conditions)!
• the argument of "this is just learning English" isn't effective - there's a difference between literature analysis and finishing day-to-day tasks.

• a product that designers make that try to solve a problem that may not actually exist, not correctly solve the problem, or overlook elements of that disability.
• when it's created, it often doesn't take into account the experiences of those who face those issues.
• learning about user research in INFO 200 right now! However, one common pitfall is when designers come in with preconceptions on how users would use their work, and only really ask questions based on the designer's own experience (rather than the user) - similar to this!
• feels like those creating disability dongles are trying to compensate for guilt/pity towards disabled people - instead of properly putting themselves in other people's shoes, and not getting the input from actual users.
• favorite example was the stair-climbing wheelchair: why not install a ramp (instead of charging people thousands of dollars)!
• instead of making something accessible in the first place, disability dongles are almost an "add-on" (that you have to pay for), which is not useful!
• real dongles are converters which exist when hardware doesn't have the necessary plug. Instead, we should try to standardize things so that a dongle isn't necessary.
• as an aside, nobody enjoys using dongles!!

• when solving a problem, figure out if there is a systematic solution or not
• one example: "smart" glasses for deaf people that display captions on the lenses sound interesting. But, this puts the burden on deaf people. Instead, systems like closed captions or sign translators fix these systemic issues, rather than forcing deaf people to help themselves.
• currently taking HCDE 315 (inclusive design) which focuses on disability as a mismatch with how the world is designed. It's the world that we should change!
• too easy to slip into "let's build technology" and build something cool, rather than actually helping people.

• AI feels too much like a buzzword - e.g. AI rice cookers?
• but, it's certainly helpful - e.g. with automatic captioning. However, it's still unreliable/inaccurate and may not solve the problem.
• in a way, feels like a dongle or a bandaid fix.
• going back to the plain text readability project: you could have a crowdsourced database that converts complicated text into plaintext. AI could do some of this, but it can miss a lot of nuance. Useful as a tool, but not the solution.
• what are we letting AI learn off of? Machine learning could be learning data from exclusionary sources or replicate biases.
• and, generative AI tools are still inaccurate!

• reasonable as an only option - better than nothing. But, not a permanent solution.
• feels like a constant "imperfect solution with issues" versus "is this perpetuating the problem" debate.
• AI is not at a place where we can trust it to perform important jobs, but AI is in a "beta" phase - and we need to collect feedback!
• throwing AI at things without monitoring its use "feels weird", e.g. image-generating AIs could be trained on other images generated by AI, creating nightmare fuel
• can't discount how big of an impact this could be: imagine if someone could select and copy-paste a paragraph into ChatGPT, and ask "summarize this concisely" or "translate this into another language" - that's a huge deal!
• need to be careful of utilitarian thinking - can be easy to fall down the rabbit hole of "if it doesn't work for everybody, we can't use it". Okay to use things even if there are some downsides.
• we should help develop what people are using - so if disabled people are using ChatGPT, we should work to make that better!

In-person, we discussed:

• implementing some legislation forcing websites to be accessible (e.g. requiring all images to have captions)
• but, people could just write "image", or add some well-intentioned (but bad) alt text.
• what would you do about legacy websites?
• creating software or languages that make it easier to be accessible (e.g. describing images by default, making it easily navigable)
• related solution: can we feed pre-existing structure from things like markdown into a website?
• teaching accessibility somewhere. But, where? Last week, we saw there are many tough questions (what age, what program, should it be required)?
• should be similar to how we teach civil engineers or architects ADA guidelines - through accreditation. Why not add it as a checkmark for a CS degree?
• for K-12, we should teach it somewhere, but maybe not the technical details (are students building websites)?
• K-12 doesn't immediately solve the problem, since you need to convince many CEOs and VPs. How long would it take for kids now to get to these positions of power?
• need some sort of workforce education; could be training programs or through representation.
• when onboarding as a research assistant, had to do mandatory modules about Title IX, OSHA, etc. - could do something similar.
• but, many people don't pay attention to mandatory trainings, and those people are the most important people to teach! You need to teach people to care!
• as a resident assistant, have to go through many accessibility trainings (over a week). But, they're quite effective - in part because they weren't just tacked on.
• in addition, there were tangible outcomes - e.g. if a poster wasn't accessible, it would be denied.
• could require government agencies to have accessible websites through law, and incentivize private companies to make accessibility a priority through tax breaks.

I also asked us all to answer this question on paper as an "exit ticket". Here are a few of the answers that touch on ideas we didn't talk about in-person.

• on a personal level, taking more initiative to add alt-text and other screen-reader friendly items for individual creations.
• teach about accessibility from a very early age - so that thinking about how to make the world accessible is something we all think about, in everything that we do. Almost making it the default!
• avoid reinventing the wheel - instead, first ask if the design is needed!
• Wordplay!

• The video uses a very simple example (3 dimensions instead of 120000), so it's not clear if computers are actually subtracting gender to assume that things are feminine or masculine.
• Using gender could lead to the wrong context: for example, "king - man = queen" ignores that queen doesn't always refer to royalty.
• This adding/subtracting directions feels a bit binary.
• You could have issues where historical biases creep in: e.g. maybe "president" is assigned a male connotation since the U.S. has not had a female president, so "subtracting" female could give you vice-president or first lady?
• In some languages (e.g. French or Spanish), nouns themselves have genders (and this is a core part of the grammar). How does this work?

• Algorithms & humans are actually quite similar: these algorithms are human-made. So, if the person who makes it has a subconscious bias, than the algorithm will too.
• Throughout U.S. history, policing has always been racist - so we can't just "fix the data" when all of the data is the problem. Even if we remove explicit categories (e.g. "race"), there are many proxies (e.g. see redlining and low-income housing).
• Algorithms are controlled, tailored, and have lots of nuance; they can be more explainable than humans (since you can't pry open a human's brain). So, a more realistic solution could be fixing our current algorithms.
• Change in human behaviour is very hard, especially since biases can be integrated into your environment (e.g. parents, school, where you're from). Also noticed that the locations discussed in the ProPublica article are more likely to be historically racist.
• The questions that COMPAS asks are inherently biased - is it even possible to tweak questions like "have your parents gone to jail?" to not have racial undertones?
• You could try to weigh algorithsm differently, but it's a delicate balance. For example, Google's Gemini tool went way too far in the opposite direction. And swings in either direction have big impacts on people's lives.
• Fundamental problem is that humans are prejudiced and we need to fix that within humans. But, since governments and companies are going to use these tools, we need to fix them anyways.
• In order to debias algorithms, you need completely unbiased data and people. But that's not possible - this is all systemic!
• Slight disagreement with previous point - do you need unbiased people to recognize and fix bias in data? We're all biased, and yet can see issues with current systems.
• Would like to see AI with a human in the loop - AI cannot have historical context or understand why data has different outcomes, but a human could!
• Related to our education discussion: colleges have essays because we don't think just an SAT/ACT score fully tells someone's story. Algorithms can't truly understand language, systemic oppression, or context!

• In everday life, we use lots of things we don't understand (like our brains or our phones). This should be no different! (but of course, like the brain, we should still try to understand it.)
• What's most important is if they help us - and in this case, these do, so we should use them.
• Tying back to computing education - those who understand computer science (and LLMs) have much more power and can make these decisions on behalf of many other people. So, we need to think about who has this knowledge (and what biases that reflects).
• If you understand that ChatGPT is a black box and can make mistakes (and is not an infallible god), it would be good to use. However, not sure if this is how people really view it.
• There is some accountability - e.g. the recent "Lying AI Chatbot" case with Air Canada.

• missing from the article: the physical cost of making the computers, like raw minerals and labour
• many of the raw minerals come from conflict zones (e.g. cobalt in the DRC)
• land use! (to host the data centers and servers)
• people who do not use these technologies still have to pay for them - e.g. people in the DRC may not even be using ChatGPT, and are being disprivileged of their own land and resources.
• climate change is regressive (i.e. marginalized communites get affected more). For example, unhoused people are disproportionately affected by climate change.
• also, climate change affects everyone! literally, everyone!

These datasets are gigantic, within a few orders of magnitude of "the entire internet". For proprietary models (like GPT-4), it's hard to get an exact number (since OpenAI has chosen not to disclose this). For GPT-3 (its precursor), OpenAI's 2020 paper says roughly 500 billion tokens from the Common Crawl dataset, various public domain books, and Wikipedia. OLMo, an open-source LLM developed in part by people at UW, uses the 3 trillion token dataset called Dolma.

Many of these datasets combine archives of written books and content produced on the internet (often gathered through "web scraping"). The internet is famously toxic, and many AI models can replicate this toxicity - Microsoft's Tay was an infamous example that was shut down within 24 hours of being released. One of the most famous papers studying this phenomena (RealToxicityPrompts) is from CSE professors at UW!

Data sources certainly create biases in machine learning models! "Bias" can mean all sorts of things - from racism, sexism, and ableism to working better for certain human languages or opinions the model may repeat. It's hard to summarize the history of bias in ML in one paragraph (or even one article), but Joy Buolamwini's TED Talk is a common entrypoint.

Avoiding bias is very challenging, and broadly speaking is an open problem in machine learning (and in computer science). To summarize the lay of the land: "just get unbiased data" is typically not feasible, and may not itself resolve the problem. Many researchers work in a subfield of AI called fair machine learning (and related ideas, such as "responsible AI"). Some approaches are purely technical (can we quantify "fairness"? can we then optimize for this metric), while others focus on transparency (e.g. "Model Cards"), representation, or regulation. Timnit Gebru (from the rabbit hole readings) is one of the leaders in this field!

Long story, short, "their best"! Like the video discusses, many of these models are based on probabilistic reasoning. Typically, models will still pick the most likely option (e.g. most likely token) rather than explicitly fail, and then continue chuggling along. Among other things, this explains how language models can output gibberish (if you give it inputs of things that don't exist)!

Great question! Generally speaking, there are two broad steps in machine learning (that involve different physical structures).

The first is "training", where the model tries to figure out the best weights (the linear algebra mentioned in the 3Blue1Brown video). Practically speaking, training is a bunch of math (in particular: matrix multiplication and some calculus + probability calculations). This is done on specialized computer chips that are really, really good at doing math; the most common example is a "Graphics Processing Unit" (GPU), which is particularly good at doing matrix multiplication (the fundamental operation for much of computer graphics and games). The exact data is not available for proprietary models (though Sam Altman has claimed that training GPT-4 was at least $100 million). Using OLMo as a proxy again, they trained their model twice: once on 1024 AMD MI250Xs (10k per chip), and once on 216 Nvidia A100s (very hard to actually buy, but also ~ 10k). Competition for buying graphics cards is fierce - this is one of the biggest "moats" established ML players have.

OLMo also publishes power consumption and carbon footprint estimates - they say that training used 239 MWh of energy, which is about the amount of energy generated by all hydroelectric dams in the Northwest United States (thanks Nathan Brunelle for finding this link!). They estimate training used about 70 tonnes of CO2, which is about 150000 miles of driving.

The second step is deploying the model - or in other words, letting people use it. Generally, this is done with servers across the country (and the world!) - every time a website is visited, a complicated set of algorithms figure out how to direct that query to a specific computer, that then tokenizes the input, does the matrix math, and then sends it back to the user. Cloud providers (such as Amazon Web Services, or AWS) own hundreds of datacenters that serve this exact purpose (and make boatloads of money). One of the most famous AWS regional datacenters is us-west-2, which is right beside us in Oregon! There is no explicitly public data on how many of these servers AWS owns, but almost every estimate puts it into the millions.

In short, not really (but it depends)! These models are so complicated (with billions of parameters), so it's not currently possible to truly "explain" what each of the individual pieces of the model are doing. This subfield of AI is broadly called "explainable" and "interpretable" AI, and is one of the most active fields of research.

However, researchers (and members of the public) can make broad explanations of how some parts of the model work - such as the explainer video that we just watched!

In short, this is really challenging! Very briefly (and reductively): people used to use hard rules to split tokens up into nouns, verbs, adjectives, and other "rules" of grammar. However, these tended to be very manual, error-prone, and not scalable (especially to other languages). Modern approaches often blend some of this domain-specific knowledge with statistical tools that try to "guess" what tokens are based on a set of data (in other words ... more machine learning). A famous (very technical) tutorial that gets quite close to the bleeding edge is Andrej Karpathy's Let's build the GPT Tokenizer.

In their reflection, a student recommended InfiniteCraft, which is a fun game that shows some of this in action. Neal also has a ton of other fun visualizations on his website!

I will mostly skip this question since it's not super relevant to our discussion, but long story short, it takes in a set of numbers, and "normalizes" them to all be between 0 and 1 (but keeping their relative "size" to each other). Why? In math, we typically define probabilities as being between 0 and 1 - so softmax lets us "shrink" large sets of numbers to be valid probabilities.

Long story short, models do lose context, overfit, and often cannot fit on cheap computers. So, lots of smart people work on this (and invent cool tricks to make it work). One of the most relevant ones (that you'll explore yourself in CSE 123) is "compression" - basically, treating the input from the user, weights, and other "big" things in the language model as things that we can put into a zip file and make smaller. If you stick around in 123, you'll learn about one of the classic ways of doing this, Huffman coding; modern linear algebra techniques like sparse matrix computation are extremely helpful.

Generally speaking, it isn't. Some existing laws that apply to technology (like intellectual property and copyright) may apply, but courts are in the process of deciding that just right now! The closest thing we have to American legislation is an executive order from the Biden administration from October 2023, but it's not yet clear how this will be enforced.

This was a great question! Unfortunately, after some digging, I couldn't find a reliable first-party source (many articles claim to know the answer, but looking through the citations left me unsatisfied).

Some things I could verify were:

• as of April 2024, OpenAI charges $30 for 1M input tokens and $60 for 1M output tokens via its GPT-4 API. The token:word ratio seems to be 4:3, so if you estimate that a query is about 50 words and a response is 250, you get a napkin math pricing of 1.4 cents per query.
• BLOOM, a comparable LLM, is estimated to generate 1.5g of CO2 per query (per a paper written by members of the BLOOM team
• estimations are quite a bit more complicated because it's hard to factor in upfront costs from purchasing hardware and needing to repair it. There are many more estimates with "amortized" costs; the controversial Nature paper The carbon emissions of writing and illustrating are lower for AI than for humans
may be of interest!

First, we did an activity based on categorizing different types of data and its privacy implications. We talked about items inspired from the survey on the right to be forgotten (medical records, financial records, criminal records, embarassing photos) and ones related to you as a student (student records, employment, and news articles about people). We rated them across two axes:

• would we be comfortable with the information being completely public, completely private (e.g. just available to you), or available to third parties upon request (with a password or some other system)
• if the information concerns a certain person, should we allow requests to delete that data?

The class bucketed different types of data in different areas! We touched on a few key points:

• "news articles" is a really tricky piece! Outside of existing processes for libel and defamation (when the article is clearly false), there's a delicate balancing act between freedom of speech, holding powerful people accountable, the right to privacy, and the fact that it's hard to write a straightforward rule for these situations. Two prescient examples we talked about were public reporting on items that people may want to hide from their public image for completely acceptable reasons (e.g. drag) and the rights of children and letting them move on in adulthood, especially for child stars or athletes. Who are we to tell people what an embarassing news article is, but also how can we contrast this with important news that the public deserves to know?
• "student records" is tricky too! We briefly talked about how "student record" is more expansive than simply your grades and homework in a class (it includes communication about you in classes, intersects with you as a student employee, and can even include private messages about students). In addition, we talked about FERPA and its provisions in requiring retention for a certain amount of time, and required provisions on deleting data.

• Q: are modern cars (maybe made in the last ten years) easily hackable? Or have they fixed the problems Yoshi talked about?
• Matt's answer: as a non-expert (but someone who has worked directly in this field, using the same technology), several things are true. Some car manufacturers have taken note and have patched some of the low-hanging fruit, while others have not (and many research and industry teams have replicated these types of attacks over the last decade - Matt's expertise is in attacks on the "CAN bus" in cars). However, even for car manufacturers that have fixed these issues, there are many, many other attacks that work against cars. Notable examples that Matt is personally familiar with include "replay attacks" on keyless cars and generally poor security for in-car entertainment systems. In many fields, security is an "arms race": security researchers or hackers will find vulnerabilities, engineers will fix them, and researchers or hackers find another angle (or exploit a bug in the "fix").
• Matt's tangetial answer: one of the things that keeps the average person generally safe is that ... it's not worth the effort to set all of this up to hack into a college student's car (especially when you can usually just break the windows and hotwire it - no hacking necessary). However, this is a bigger deal for higher-profile targets, often in the lens of national security.
• Matt's post-hoc followup: if you are interested, one interesting profile of the front-to-back of car hacking is the Wired article "Hackers Remotely Kill a Jeep on the Highway—With Me in It" by Andy Greenberg (2015).
• Q: how easy is it to hack into a "smart home", and how could that work?
• Matt's answer: it's definitely possible! An infamous case is hacking into smart lightbulbs (that has reached almost meme status). There are many different approaches to take, but one of the simplest is to find something connected to the internet (e.g. a smart garage door, fridge, or lightbulb) that has bad "access controls" or other security (e.g. using the default username and password for the device, basing the password off of serial number, or just ... not having a password). After you gain access, you can then wreak all sorts of havoc! Similar to my answer about cars, there's almost an "arms race" of security here too; companies have started to patch the "low-hanging fruit", but researchers and hackers have just moved on to other techniques.
• Matt's post-hoc followup: I forgot to mention in class, but companies pay serious money to researchers who find these bugs and report them (rather than using them in the wild). For example, Apple's Security Bounty can pay upwards of millions of dollars for researchers who find these types of nasty bugs.
• Q: what is the "safest car"?
• Matt's answer: I don't have a specific make or model in mind, but there's probably a balance of minimizing your "attack surface" (i.e. things that are hackable) - so perhaps, no infotainment system, no wireless tracking, no keyless fobs, etc. - while still being modern-ish so that there is a layer of security over the computers that power all cars (e.g. the CAN bus). This would be a great question for Yoshi and other researchers!
• Q: Yoshi's video mostly talked about the brakes - could you blow up the engine? How does this work with other things (like boats or planes)?
• Matt's answer: in short, probably yes (this is what I worked on as an intern - applying these concepts to naval ships). The "CAN bus" system (fancy word: network topology and protocol) is often used in ships and planes. You can disable the thermal regulator (or coolant dispenser, temperature monitor, etc.) on some engines via a CAN bus attack, which can cause the engine to predictably overheat and enter a failure state. However, the specifics are quite complicated (and perhaps best suited in a different type of conversation).

• There are so many technologies that use lots of math, are related to security, and we have no idea how they work (e.g. Face ID) - we just trust that they keep our information safe.
• Things like this YouTube video help though, and we should have more of these things!
• Would making the implementation open help? Maybe that would cause security issues because people can see exactly how it works...
• But, making it open also makes it easier for all of us to trust the code and audit it for bugs. This is a common argument against security through obscurity.
• Also a difference between trusting the overall idea in theory, and implementations in practice - it might have a bug, people may not prioritize it, etc.
• Would be good to have experts review and approve these solutions - almost like toothpaste commercials?
• How would we find these experts? Who is your ideal panel?
• Would be good if they are public and can be held accountable (e.g. for toothpaste commercials, who are the 4 out of 5 dentists)? And, they should be independent from whoever is implementing this security (and should not be paid by them).
• But also, people can be pretty suspicious of the government? And many of these explanations might involve complicated math that most computer science majors wouldn't understand (let alone the public).
• [at this point, Matt briefly chimed in that this exact concern has happened with security! If you're interested, you may be interested in Daniel J. Bernstein, who is famous for a Supreme Court case on cryptography and arms export regulations and his work on the Dual_EC_DRBG "backdoor", among many other contributions to computer science]
• Related note: the video was sponsored by (and done in conjunction) with the US Census Bureau - those were the experts!

• A good baseline would just be to give out the code and math itself.
• But, for the general public, it'd be pretty challenging to understand. Maybe we need to break things down more?
• One analogy could be nturition facts?
• But, much of the limiting factor here is higher education - requires much more subject matter expertise (math & computer science) than a nutrition facts label.
• And, there are people who can't fully comprehend a nutrition facts label!
• Related point: think of the foods that advertise with "only with ingredients you can pronounce" - do we want that for CS?
• Is the average person even aware that they need to be concerned about data security? This class is not representative of the "average" American (since we've all taken a computer science class). Maybe we need to make people more aware of security first.
• Related to last week's reading on "Machine Bias", wasn't the company not required to disclose how the algorithm exactly worked?
• Maybe it doesn't matter too much because, even if the average person doesn't understand it, they'll still use it.
• The math part of the security is not even that important. For example, 2-factor authentication is pretty common, but now people use SIM swapping to hack into celebrity Instagram accounts - better encryption doesn't fix those!
• Many (most?) hacks involve attacking the person ("social engineering") - e.g. spearfishing, "Nigerian prince scam" - and that's what we need to fix, with better education.
• The math issues might be more of an issue for companies like Google or bigger/more important people.

Where have you seen laws of design in apps that you use?

• Listed apps: Spotify (2x), Instagram (2x), TikTok (2x), BeReal, Word, Google Docs, Gmail, VLC, Canvas, Ghost Commander
• Big emphasis on consistent patterns across apps - e.g. the "heart" icon meaning "like" across various social media sites, and expecting similar features (and keyboard shortcuts) across Word and Google Docs
• A note that many social media apps restrict the number of things you can do, perhaps to minimize cognitive overhead (but also can be frustrating!)
• A note that these design norms are almost a part of competition - people won't use your Word competitor (e.g. Notion, Obsidian, Bear, Evernote, ...) if it's too different in design!

Where have you seen dark patterns?

• Listed patterns: hidden costs, fake limited-time offers, microtransactions, loyalty programs and streaks, subliminal and hidden advertisements, guilt (Wikipedia donations, Duolingo), tipping
• Discussed how Duolingo has really committed to the guilt bit (including creating many joke advertisements), and how that contrasts with more serious situations (e.g. guilt would probably be viewed poorly in a college setting)
• A discussion on different types of dark patterns with tipping: from it being a hidden cost, to guilt around making sure employees are paid, to the physical interface of tipping machines (e.g. having pre-selected options being too high, making it hard to make a custom or no tip, etc.)

Where have you seen unwanted innovation?

• general listed topics: disability dongles, "adding AI to everything", requiring an app when it's not necessary (e.g. Disney, parking), forcing algorithmic timelines, changing layouts in apps (mentioned: AirDrop, Google, Instagram)
• brief discussion on how this is almost exactly the same as the disability dongle article, and how this is a common issue with engineering - building cool things without asking if it's needed!

• Don't think there should be laws: "follow these things always" can stifle creativity, and you wouldn't be able to adapt to different situations.
• e.g. look at art history: some of the most interesting art breaks existing artistic laws and norms
• While innovation is great, having guidelines to make websites similar to previous ones is good - getting rid of these "laws" could harm usability.
• These laws could be great for teaching and getting started, especially when design is so open-ended.
• Think of them as guidelines. Some are just good design (e.g. "don't put white text on a white background"), but generally speaking you can break some of these laws and still have good looking things!
• Keep in mind that "intuition" is different for each person, and you want to be more general/intentional about thinking about what is intuitive.
• The answer depends on the context and your goal: if your goal is usability for a broad audience, you may want to prioritize design laws and intuition; but, if your goal is to create web art, ignoring the laws can help you be more creative.
• If you follow the laws of design, you may feel like you have to follow existing products (like Word) - but that could be stagnant, and then you'll always have Word.
• Need to strike a balance: if something is too foreign, people won't use it!
• Related to the history of the printing press and movable type: for more, see the Gutenberg Bible.

• In photography, one of the first things you focus on is the "rule of thirds" and prioritizing symmetry and the golden ratio. But, some of the most breathtaking images come from exceptions.
• Similar to genre conventions in music: pop songs can be formulaic and follow a similar format or unspoken structure. But, songs that really break the mold are more memorable!
• Literature: some of the best books can be very nontraditional, but things that stand the test of time / the literary canon are books.
• Sometimes, breaking norms doesn't work and fails spectacularly: for example, early 2000s websites were very hard to navigate! Now, the web is more standardized (if also boring), which can make it more accessible.

In Amy's article, she talks about how programming languages are a human interface - which would make code quality a potential law of design!

Broadly speaking, there are three types of code quality rules in 12X:

1. Rules that are almost universally agreed upon, similar to "don't put white text on a white background". Indentation is the prototypical example here: almost all programmers (Java or not) follow indentation rules.
2. Rules that are controversial, but we pick an option to make things consistent (and thus, more usable across the class). A classic example is how many spaces to indent by: many people disagree on 2 versus 4 spaces, whether or not you should use tabs or spaces, etc. Another is commenting style: there are many, many different approaches to commenting, but we've picked BERP & pre/post to provide focus.
3. Rules that exist for pedagogical purposes (e.g. "forbidden features") - mostly so that you learn a specific skill or pattern, rather than skipping over it.

But, sometimes we break the rules of 12X code quality!

• With "BERP", the "E" (exceptions) is frequently excluded when it's not necessary.
• Context matters: for quick tests, we often don't comment our code or pick bad variable names; but for final submissions and code we'll share with others, we care more about these.
• When instructors live code, we often don't follow all the code quality guidelines (e.g. commenting) - in part because the purpose is different!

• Definition 1: someone convincing you to do soething you didn't want to do. But, what about ... homework? Or requiring a user to sign in?
• Definition 2: intentionally (or indirectly) misleading someone into doing something. But, what does indirectly mean? What about long terms and conditions?
• Definition 3: influencing or manipulating the consumer beyond the scope of advertising. But, isn't some advertising fine? What about the method of advertising?
• Definition 4: "obscuring" the truth as a result of manipulating the consumer's lack of information or knowledge. But, what about things like infinite scroll?
• Maybe we only apply these laws to larger companies?
• Need to avoid getting to the state of really long terms and conditions - people don't read them! This is kind of like TikTok's "you've been scrolling for a while" video - most people just keep on scrolling.
• Does Steam's "you've played __ game for __ hours" help or hurt? On one hand, it's very in-your-face and not deletable; on the other hand, it could be a badge of honour.
• What if we had lifelong screen time? Confronting the user can be very powerful!
• But, isn't that guilt or shame? And users can get used to pressing "ignore" every time?

From Matt: most students answered that copyright law should apply to AI-generated content, and the medium doesn't matter too much.

As a rebuttal: one common talking point in the AI + copyright conversation is that "humans who look at art and then are inspired by it are not beholden to copyright. LLM training is just like reading all the English books before writing your own. Why is AI any different?". Do you agree or disagree?

• Disagree: the core difference between a human and AI is creativity. Humans can invent beyond what they've seen, while AI cannot.
• The tension could be more the lack of giving credit rather than the actual act of training and/or generating copied images.
• When you learn art, there is a social contract based on respect for the artist and their work. A company training an AI model (and not giving credit) is not paying respect - the actual training process is red herring.
• There are copyright laws that exist now that could essentially apply to generative AI - e.g. requiring AIs to cite its sources, or for training to do so.
• AI isn't just purely copying and reproducing work - it is changing it slightly (keyword: "transformative work".
• Sidebar: could we train an AI to cite its sources? (Matt: theoretically possible, but as of now, practically infeasible)
• In pop music, there are also many situations where music sounds the same (e.g. Olivia Rodrigo and Taylor Swift's Cruel Summer). But, in music, there are some nuances about copyright (e.g. homages, sampling, covers). The core issue there (and here) is credit - you need to attribute work when necessary.
• Copyright law in general is very messy! If you watch Nathan For You, a savvy Canadian businessman does a lot of fun things with business -- including "dumb Starbucks" (which was allowed because it is parody). Also, many YouTubers complain about copyright law - e.g. is reacting to videos "transformative work"? Should there be copyright strikes for reaction videos?
• It's hard to prove copying. For example, many people rip off Mr Beast's video style, but it's hard to prove that they're actually coping off of him. With AI, you could prove it since the data is literally there.
• How related is inspiration and memory? ChatGPT can't actually completely memorize everything - it's using its data more like a stencil.
• How different is humans being inspired by things and AI extracting data? What actually is creativity??
• At the end of the day, AI is just a tool - people are the ones using AI to break copyright law, etc.
• Humans copying things requires skill and dedication - but an AI can do it effortlessly. Similarly, a human prompting an AI takes no skill.
• Disagree: prompting an AI is actually hard!
• For music covers, the artist needs to okay it - can we do the same here?

• Could we build AI that can explain "generally" what data influenced their response (e.g. 50% of this answer comes from X source, 50% comes from Y), and then mandate that AI companies properly label their data?
• Can we make something like Turnitin for AI-generated art?
• tools like this already exist - Turnitin has one, but its accuracy is dubious, and startups like GPTZero exist, but their accuracy is also controversial
• separately, there's the idea of "watermarking" AI-generated output; the challenge is that a user can just remove the watermark and/or spoof it
• post-hoc note from Matt: on the day of our session, Google announced a new video generation model called Veo, which uses a watermarker called SynthID
• Similar to what we'd expect in a school essay, we can require AI to cite "non-obvious" or "not common-sense" information (heuristic: 5+ sources say something is true, then it's common-sense).
• But, how would we define common sense? This would differ across cultures. And, is 5 sources enough?
• What if we treat datasets as opt-in (rather than opt-out); as an artist, you need to agree that you want your work to be trained on. But, not sure how we'd enforce this (certainly that is not happening right now)...
• Can we treat this like copyright infringement on Etsy? e.g. if someone uploads obviously copyrighted or trademarked stuff, take it down?
• this is very hard to do reliably, and typically only very big companies (like Disney and Nintendo) have the money to do this.
• but, Disney and Nintendo can also be tools! For example, there are Twitter bots that make t-shirt stores using other people's art. Artists have figured out that by posting a Mickey Mouse artwork (with the text "I am trying to infringe on Disney's copyright"), Disney will instantly start legal action and take those bots down. What if we do something similar with AI?
• right now, there already are huge lawsuits like this - like the ones we read about in the readings!!
• tangent: nintendo and disney are really harsh with copyright enforcement...

• initial thought: "fighting harm with harm" is bad. But, it's more complicated than that...
• is fighting corporate greed with harm okay? Especially as companies have money for legal fights, while individual artists do not?
• nightshade actually makes the art worse (especially as you turn up its parameters) - and you won't be able to beat big tech in an arms race of detection and catching.
• it shouldn't be the artists' job to stop their work from being stolen. If a company is "changing the world", then they definitely should have the money and resources to make sure that they aren't stealing people's work (or cite sources)! Corporations need to re-evaluate and figure out how to act ethically...
• the perpetual arms race (with an "antidote" and nightshade) could also cause long-term harms with the usage of AI, since they could learn the "wrong" things - which could hurt people or stall the technology. And, this would also make new Nightshade art worse.
• similar example: captchas have gotten more distorted as AI has gotten better, and as a result, a third party (humans filling out the captchas) are also hurting.
• relating to the reading: imagine if a person with a disability impacting their vision (or visualizing things) relies on generative AI to recognize items in pictures. If their tool spits out the wrong information because of Nightshade, we are hurting accessibility use-cases of the tool.
• the conversation should not be about "fighting" GenAI, but rather creating policy - can we do something similar to the GDPR/CCPA with cookie consent notifications, which require users to opt-in to companies using their data?
• or, something like the "search by creative commons" option on Google?
• it might not be fair/moral to use nightshade on your work if someone wants to use it under fair use - e.g. for education and nonprofit use. If it's socially beneficial, then maybe this should be okay?
• agree with the fair use point - this feels like putting malware on a library computer (or public space) just because people can misuse it.
• feels dystopian: if you put art out in the world, it's not your obligation to make sure that it doesn't hurt people when people grossly misuse it - then, you couldn't do anything!!

• even though it is legally okay, it feels morally wrong: you're trying to compete against Elastic, and taking their main product.
• the fact that it retains a name containing "Elastic" felt particularly bad (maybe IP issues?) - especially since it harms their ability to market.
• feels strange that Amazon claimed it was a "parternship" with Elastic, when they are also competitors.
• but, Amazon did contribute engineering resources to the core product. Does that count as parternship?
• even if Amazon contributes engineering resources, the power imbalance and size of Amazon makes this an issue - Amazon doesn't need to compete?
• would it be more okay if a startup did the same thing with Elastic?
• well, Elastic (the company) still seems to be doing fine - at least the CEO is worth lots of money :)
• disallowing this sort of "copying" goes against the ethos of open-source - even though this specific instance feels shady, this is exactly what open-source is supposed to allow.
• Elastic did forfeit their rights, and Amazon is a company that is trying to earn as much money as possible - so within these bounds, perhaps moral?
• Amazon broke the social contract around open source (this is why we can't have nice things). But, if we add laws to restrict open-source, this could hurt the world,
• could we instead innovate with open-source licenses to prevent behaviour like this? We already have an existing legal framework to deal with licensing.
• what are our thoughts on the GPL and other "copyleft licenses"? (one key clause of the GPL family is that all derivative works need to be distributed with a similar license, i.e. open-source)
• open-source is supposed to be mutually beneficial, and it sounds like the GPL preserves that more: the author helps the world by making their code available, and the GPL requires the world to contribute back too.
• the GPL could force "bad faith" actors like Amazon to contribute back and be more mutually beneficial

• no: it's just your own project, and you don't owe other people anything just because they're using your code.
• you shouldn't be bound to a project just because you worked on it for a moment: you have a life and should be able to prioritize other things!
• the point of open-source is collaboration: putting the blame and responsibility on one person seems counterintuitive. If there's a problem, you could go fix it (instead of making the creator do so)!
• but, what about external harms? In other situations - like when a company makes a product - if they release something and it hurts people, they're on the hook (for at least negligence). Why would that not apply here?
• in open-source, you're not working for a company (which has different moral and ethical implications) - you're working for yourself. You owe the world less!
• companies profit off of manufacturing - so, it then makes sense to hold them accountable (with fines, laws, etc.). Open-source maintainers aren't benefitting from it in the same way.
• we can hold the consumers of open-source software accountable: if a company makes a product using open-source software hurts people, it's on the company for not double-checking the code (and being negligent)!
• someone could abuse this "open-source software has no liability" loophole.
• consumer expectations are important: similar to volunteering (where you may have lower expectations because you know they're a volunteer), we could treat open-source the same way.

• interfaces seem more comparable to a framework, structure, or strategy - it's different from copying someone's implementation.
• if interfaces are protected this way, this could harm the actual utility of the interface.
• the point of interfaces is to make compatible software!
• but, the interfaces discussed in the lawsuit are probably much more complicated than the ones we write in CSE 122. It probably required a lot of thought - which is the thing that we should protect?
• interesting analogy to recipes (when they're just a list of ingredients/instructions), which are generally not patentable/copyrightable.

• it feels like algorithms shouldn't be patented - isn't the whole point that everybody can then use them, implement them, and improve upon them?
• but, the point of patents is for that exact reason - that's why you have to explain how the thing works to patent it.
• how would you differentiate minor changes in an algorithm?
• also, there are different types of patents (e.g. patenting a process is different from a broader idea). And, there are some restrictions (novelty, non-obvious).
• algorithms also feel like a strategy rather than a tangible thing - it's not like making a tool or physical object. Can you copyright something like this (like folding in as a technique?
• related to math: it'd be really bad if someone patented or copyrighted Newton's equations, Gaussian elimination, or the idea of standard deviation.