Skip to content

Commit e0cb980

Browse files
kcaylorkcaylor
committed
docs(update): added clarify about series index and dataframes
1 parent 66f9fa6 commit e0cb980

5 files changed

Lines changed: 106 additions & 38 deletions

File tree

‎course-materials/cheatsheets/data_merging.qmd‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -401,5 +401,3 @@ result_all = pd.merge(df1, df2, on='key', how='outer')
401401
print("All possible combinations:")
402402
print(result_all)
403403
```
404-
405-
Remember: Always examine your data before and after merging to ensure the operation worked as expected. When in doubt, start with a small sample of your data to test your merge logic!

‎course-materials/interactive-sessions/4a_dataframes.qmd‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -235,6 +235,32 @@ long_named_cities = cities_df.nlargest(5, 'name_length')
235235
print(long_named_cities[['name', 'country', 'name_length']])
236236
```
237237

238+
#### Adding Series as Columns: Index Alignment
239+
240+
When adding a Series as a new column, pandas aligns data by index:
241+
242+
```{python}
243+
#| echo: true
244+
245+
# This works - Series index matches DataFrame index
246+
sample_series = pd.Series([1, 2, 3], index=[0, 1, 2])
247+
small_df = pd.DataFrame({'A': ['x', 'y', 'z']})
248+
small_df['B'] = sample_series
249+
print("Aligned correctly:")
250+
print(small_df)
251+
252+
# This creates unexpected results - misaligned indices
253+
misaligned_series = pd.Series([10, 20, 30], index=['a', 'b', 'c'])
254+
small_df['C'] = misaligned_series # Results in NaN values!
255+
print("\nMisaligned indices:")
256+
print(small_df)
257+
258+
# Solution: Use .reindex() or ensure matching indices
259+
small_df['D'] = misaligned_series.reindex(small_df.index, fill_value=0)
260+
print("\nCorrected alignment:")
261+
print(small_df)
262+
```
263+
238264
## 7-8: Basic _Grouping_ and _Aggregation_
239265

240266
Grouping allows us to perform operations on subsets of the data:

‎docs/course-materials/cheatsheets/data_merging.html‎

Lines changed: 13 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -466,7 +466,7 @@ <h2 class="anchored" data-anchor-id="introduction">Introduction</h2>
466466
<section id="setup" class="level2">
467467
<h2 class="anchored" data-anchor-id="setup">Setup</h2>
468468
<p>Let’s start by importing pandas and creating sample datasets to work with:</p>
469-
<div id="fc3c5ca5" class="cell" data-execution_count="1">
469+
<div id="d9e4f656" class="cell" data-execution_count="1">
470470
<details open="" class="code-fold">
471471
<summary>Code</summary>
472472
<div class="sourceCode cell-code" id="cb1"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="im">import</span> pandas <span class="im">as</span> pd</span>
@@ -514,7 +514,7 @@ <h2 class="anchored" data-anchor-id="types-of-joins">Types of Joins</h2>
514514
<section id="inner-join-default" class="level3">
515515
<h3 class="anchored" data-anchor-id="inner-join-default">Inner Join (Default)</h3>
516516
<p>Returns only rows that have matching keys in both DataFrames.</p>
517-
<div id="53fcbdbf" class="cell" data-execution_count="2">
517+
<div id="5d16a550" class="cell" data-execution_count="2">
518518
<details open="" class="code-fold">
519519
<summary>Code</summary>
520520
<div class="sourceCode cell-code" id="cb3"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb3-1"><a href="#cb3-1" aria-hidden="true" tabindex="-1"></a><span class="co"># Inner join - only students with grades</span></span>
@@ -534,7 +534,7 @@ <h3 class="anchored" data-anchor-id="inner-join-default">Inner Join (Default)</h
534534
<section id="left-join" class="level3">
535535
<h3 class="anchored" data-anchor-id="left-join">Left Join</h3>
536536
<p>Returns all rows from the left DataFrame and matched rows from the right DataFrame.</p>
537-
<div id="3ba82d28" class="cell" data-execution_count="3">
537+
<div id="b4caeeef" class="cell" data-execution_count="3">
538538
<details open="" class="code-fold">
539539
<summary>Code</summary>
540540
<div class="sourceCode cell-code" id="cb5"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb5-1"><a href="#cb5-1" aria-hidden="true" tabindex="-1"></a><span class="co"># Left join - all students, with grades where available</span></span>
@@ -556,7 +556,7 @@ <h3 class="anchored" data-anchor-id="left-join">Left Join</h3>
556556
<section id="right-join" class="level3">
557557
<h3 class="anchored" data-anchor-id="right-join">Right Join</h3>
558558
<p>Returns all rows from the right DataFrame and matched rows from the left DataFrame.</p>
559-
<div id="d05b7478" class="cell" data-execution_count="4">
559+
<div id="a939b7b1" class="cell" data-execution_count="4">
560560
<details open="" class="code-fold">
561561
<summary>Code</summary>
562562
<div class="sourceCode cell-code" id="cb7"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb7-1"><a href="#cb7-1" aria-hidden="true" tabindex="-1"></a><span class="co"># Right join - all grades, with student info where available</span></span>
@@ -578,7 +578,7 @@ <h3 class="anchored" data-anchor-id="right-join">Right Join</h3>
578578
<section id="outer-join" class="level3">
579579
<h3 class="anchored" data-anchor-id="outer-join">Outer Join</h3>
580580
<p>Returns all rows from both DataFrames, filling missing values with NaN.</p>
581-
<div id="bb2877ad" class="cell" data-execution_count="5">
581+
<div id="a17d9ad5" class="cell" data-execution_count="5">
582582
<details open="" class="code-fold">
583583
<summary>Code</summary>
584584
<div class="sourceCode cell-code" id="cb9"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb9-1"><a href="#cb9-1" aria-hidden="true" tabindex="-1"></a><span class="co"># Outer join - all students and all grades</span></span>
@@ -603,7 +603,7 @@ <h3 class="anchored" data-anchor-id="outer-join">Outer Join</h3>
603603
<section id="merging-with-different-column-names" class="level2">
604604
<h2 class="anchored" data-anchor-id="merging-with-different-column-names">Merging with Different Column Names</h2>
605605
<p>When the key columns have different names in each DataFrame:</p>
606-
<div id="1ae9cedb" class="cell" data-execution_count="6">
606+
<div id="3aac571a" class="cell" data-execution_count="6">
607607
<details open="" class="code-fold">
608608
<summary>Code</summary>
609609
<div class="sourceCode cell-code" id="cb11"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb11-1"><a href="#cb11-1" aria-hidden="true" tabindex="-1"></a><span class="co"># Create DataFrame with different column name</span></span>
@@ -635,7 +635,7 @@ <h2 class="anchored" data-anchor-id="merging-with-different-column-names">Mergin
635635
<section id="joining-on-index" class="level2">
636636
<h2 class="anchored" data-anchor-id="joining-on-index">Joining on Index</h2>
637637
<p>When you want to combine DataFrames based on their index (row labels), use the <code>.join()</code> method - it’s much simpler than merging:</p>
638-
<div id="92d18a9e" class="cell" data-execution_count="7">
638+
<div id="0d0dbe4c" class="cell" data-execution_count="7">
639639
<details open="" class="code-fold">
640640
<summary>Code</summary>
641641
<div class="sourceCode cell-code" id="cb13"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb13-1"><a href="#cb13-1" aria-hidden="true" tabindex="-1"></a><span class="co"># Create DataFrames with datetime indices (common in time series data)</span></span>
@@ -688,7 +688,7 @@ <h2 class="anchored" data-anchor-id="concatenating-dataframes">Concatenating Dat
688688
<p>Use <code>pd.concat()</code> to stack DataFrames vertically or horizontally:</p>
689689
<section id="vertical-concatenation-stacking-rows" class="level3">
690690
<h3 class="anchored" data-anchor-id="vertical-concatenation-stacking-rows">Vertical Concatenation (Stacking Rows)</h3>
691-
<div id="86096f06" class="cell" data-execution_count="8">
691+
<div id="fc92deb7" class="cell" data-execution_count="8">
692692
<details open="" class="code-fold">
693693
<summary>Code</summary>
694694
<div class="sourceCode cell-code" id="cb15"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb15-1"><a href="#cb15-1" aria-hidden="true" tabindex="-1"></a><span class="co"># Create additional data for different years</span></span>
@@ -723,7 +723,7 @@ <h3 class="anchored" data-anchor-id="vertical-concatenation-stacking-rows">Verti
723723
</section>
724724
<section id="horizontal-concatenation-adding-columns" class="level3">
725725
<h3 class="anchored" data-anchor-id="horizontal-concatenation-adding-columns">Horizontal Concatenation (Adding Columns)</h3>
726-
<div id="fc2f4103" class="cell" data-execution_count="9">
726+
<div id="0965d73a" class="cell" data-execution_count="9">
727727
<details open="" class="code-fold">
728728
<summary>Code</summary>
729729
<div class="sourceCode cell-code" id="cb17"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb17-1"><a href="#cb17-1" aria-hidden="true" tabindex="-1"></a><span class="co"># Create separate DataFrames with same index</span></span>
@@ -749,7 +749,7 @@ <h3 class="anchored" data-anchor-id="horizontal-concatenation-adding-columns">Ho
749749
<h2 class="anchored" data-anchor-id="real-world-examples">Real-World Examples</h2>
750750
<section id="example-1-eurovision-analysis-from-eod-day6" class="level3">
751751
<h3 class="anchored" data-anchor-id="example-1-eurovision-analysis-from-eod-day6">Example 1: Eurovision Analysis (from eod-day6)</h3>
752-
<div id="f4d4eecf" class="cell" data-execution_count="10">
752+
<div id="80539753" class="cell" data-execution_count="10">
753753
<details open="" class="code-fold">
754754
<summary>Code</summary>
755755
<div class="sourceCode cell-code" id="cb19"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb19-1"><a href="#cb19-1" aria-hidden="true" tabindex="-1"></a><span class="co"># Simulate the Eurovision data merging scenario</span></span>
@@ -781,7 +781,7 @@ <h3 class="anchored" data-anchor-id="example-1-eurovision-analysis-from-eod-day6
781781
</section>
782782
<section id="example-2-climate-data-analysis-from-6b_advanced_data_manipulation" class="level3">
783783
<h3 class="anchored" data-anchor-id="example-2-climate-data-analysis-from-6b_advanced_data_manipulation">Example 2: Climate Data Analysis (from 6b_advanced_data_manipulation)</h3>
784-
<div id="8c8eaefa" class="cell" data-execution_count="11">
784+
<div id="7bcee71b" class="cell" data-execution_count="11">
785785
<details open="" class="code-fold">
786786
<summary>Code</summary>
787787
<div class="sourceCode cell-code" id="cb21"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb21-1"><a href="#cb21-1" aria-hidden="true" tabindex="-1"></a><span class="co"># Create simple temperature and CO2 data</span></span>
@@ -815,7 +815,7 @@ <h3 class="anchored" data-anchor-id="example-2-climate-data-analysis-from-6b_adv
815815
<h2 class="anchored" data-anchor-id="handling-common-issues">Handling Common Issues</h2>
816816
<section id="missing-values-after-merge" class="level3">
817817
<h3 class="anchored" data-anchor-id="missing-values-after-merge">Missing Values After Merge</h3>
818-
<div id="3d14b3df" class="cell" data-execution_count="12">
818+
<div id="14b81aaa" class="cell" data-execution_count="12">
819819
<details open="" class="code-fold">
820820
<summary>Code</summary>
821821
<div class="sourceCode cell-code" id="cb23"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb23-1"><a href="#cb23-1" aria-hidden="true" tabindex="-1"></a><span class="co"># Create DataFrames with some non-matching records</span></span>
@@ -861,7 +861,7 @@ <h3 class="anchored" data-anchor-id="missing-values-after-merge">Missing Values
861861
</section>
862862
<section id="duplicate-column-names" class="level3">
863863
<h3 class="anchored" data-anchor-id="duplicate-column-names">Duplicate Column Names</h3>
864-
<div id="214b4245" class="cell" data-execution_count="13">
864+
<div id="110588fe" class="cell" data-execution_count="13">
865865
<details open="" class="code-fold">
866866
<summary>Code</summary>
867867
<div class="sourceCode cell-code" id="cb25"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb25-1"><a href="#cb25-1" aria-hidden="true" tabindex="-1"></a><span class="co"># DataFrames with overlapping column names </span></span>
@@ -1037,7 +1037,6 @@ <h2 class="anchored" data-anchor-id="common-troubleshooting">Common Troubleshoot
10371037
<span id="cb31-12"><a href="#cb31-12" aria-hidden="true" tabindex="-1"></a>result_all <span class="op">=</span> pd.merge(df1, df2, on<span class="op">=</span><span class="st">'key'</span>, how<span class="op">=</span><span class="st">'outer'</span>)</span>
10381038
<span id="cb31-13"><a href="#cb31-13" aria-hidden="true" tabindex="-1"></a><span class="bu">print</span>(<span class="st">"All possible combinations:"</span>)</span>
10391039
<span id="cb31-14"><a href="#cb31-14" aria-hidden="true" tabindex="-1"></a><span class="bu">print</span>(result_all)</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
1040-
<p>Remember: Always examine your data before and after merging to ensure the operation worked as expected. When in doubt, start with a small sample of your data to test your merge logic!</p>
10411040

10421041

10431042
</section>

0 commit comments

Comments
 (0)