Being able to factor different mathematical expressions is a common theme throughout all of algebra.   Most of these topics are discussed in an Algebra 1 course, but are given here for a nice review.   The area model shown below is a really nice visual that helps explain the factoring process.
Watch the video below on finding the greatest common factor, or GCF.
Here is another video with a slightly different method called the Area Model.
It doesn't matter which method you use to factor out GCFs.   Use the method that makes most sense and is comfortable to you.
One of the most fundamental aspects to factoring is the process called factoring by grouping.   The occurs when the polynomial has four terms (could be 6 or 8, etc.).   The process is fairly simple, and the general degree three term will serve as an example: $$ax^3+bx^2+cx+d$$
| Group the first two terms together and then group the last two terms together: | \(\color{blue}{(}ax^3+bx^2\color{blue}{)}+\color{blue}{(}cx+d\color{blue}{)}\) |
| Factor out the GCF from each group: | \(g_1x^2\left(a_1x+b_1\right)+g_2\left(c_1x+d_1\right)\) |
| The "left over" parenthesis should be the same \(a_1=c_1\) and \(b_1=d_1\): | \(g_1x^2\color{green}{\left(a_1x+b_1\right)}+g_2\color{green}{\left(c_1x+d_1\right)}\) |
| This is another GCF that can be factored out: | \(\color{green}{\left(a_1x+b_1\right)}\cdot\left(g_1x^2+g_2\right)\) |
Factor:   \(9x^2y+24x-15xy^2-40y\)
| Put into groups: | \(\color{blue}{(9x^2y+24x)}+\color{green}{(-15xy^2-40y)}\) |
| Factor out the GCF of each group: | \(\color{blue}{3x}{(3xy+8)}+\color{green}{-5y}{(3xy+8)}\) |
| The left over part should be the same: | \(\color{blue}{3x}\color{magenta}{(3xy+8)}\color{green}{-5y}\color{magenta}{(3xy+8)}\) |
| This is the new GCF, factor it out: | \(\color{magenta}{(3xy+8)}\cdot\left(\color{blue}{3x}\color{green}{-5y}\right)\) |
| Final factored answer: | \(\left(3xy+8\right)\left(3x-5y\right)\;\color{green}{\checkmark}\) |
It might be necessary to reorder the terms to get things to work out.   Consider the following example: \(8+9y^4-6y^3-12y\).   If groups are formed and then GCFs are found, this problem gets stuck:
$$\begin{array}{l} \color{blue}{(8+9y^4)}+\color{green}{(-6y^3-12y)} \\ \\ \color{blue}{1}\left(8+9y^4\right)+\color{green}{-6y}\left(y^2+2\right) \end{array}$$The "left over" portion is not the same: \(8+9y^4\ne y^2+2\).   This means that the original problem needs to have terms reordered so that after the GCFs have been factored out, the left over portion would be the same.   Which terms need to be reordered?   There is no rule here, so just reorder and see what happens:
$$\begin{array}{l} 8+9y^4-6y^3-12y \\ \\ 8-6y^3+9y^4-12y \\ \\ \color{blue}{(8-6y^3)}+\color{green}{(9y^4-12y)} \\ \\ \color{blue}{2}\left(4-3y^3\right)+\color{green}{3y}\left(3y^3-4\right) \end{array}$$In this reordering, the left over parts look close to being the same, but it appears a negative is involved.   If the GCF of the second group included that negative, \(-3y\), the left over portions would be equal: $$\color{blue}{2}\left(\color{magenta}{4-3y^3}\right)+\color{green}{-3y}\left(\color{magenta}{-3y^3+4}\right)$$ and the correct factored form would be: $$\left(2-3y\right)\left(4-3y^3\right)\;\color{green}{\checkmark}$$