Commit 944bc9cc authored Nov 26, 2019 by Marco Elver Committed by Paul E. McKenney Jan 07, 2020

asm-generic/atomic: Use __always_inline for fallback wrappers



Use __always_inline for atomic fallback wrappers. When building for size
(CC_OPTIMIZE_FOR_SIZE), some compilers appear to be less inclined to
inline even relatively small static inline functions that are assumed to
be inlinable such as atomic ops. This can cause problems, for example in
UACCESS regions.

While the fallback wrappers aren't pure wrappers, they are trivial
nonetheless, and the function they wrap should determine the final
inlining policy.

For x86 tinyconfig we observe:
- vmlinux baseline: 1315988
- vmlinux with patch: 1315928 (-60 bytes)

Suggested-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Marco Elver <elver@google.com>
Acked-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>

parent c020395b

include/linux/atomic-fallback.h

+171 −169

File changed.

Preview size limit exceeded, changes collapsed.

scripts/atomic/fallbacks/acquire

+1 −1

Original line number	Original line	Diff line number	Diff line
	cat <<EOF		cat <<EOF
	static inline ${ret}		static __always_inline ${ret}
	${atomic}_${pfx}${name}${sfx}_acquire(${params})		${atomic}_${pfx}${name}${sfx}_acquire(${params})
	{		{
	${ret} ret = ${atomic}_${pfx}${name}${sfx}_relaxed(${args});		${ret} ret = ${atomic}_${pfx}${name}${sfx}_relaxed(${args});

scripts/atomic/fallbacks/add_negative

+1 −1

Original line number	Original line	Diff line number	Diff line
	@@ -8,7 +8,7 @@ cat <<EOF
	* if the result is negative, or false when		* if the result is negative, or false when
	* result is greater than or equal to zero.		* result is greater than or equal to zero.
	*/		*/
	static inline bool		static __always_inline bool
	${atomic}_add_negative(${int} i, ${atomic}_t *v)		${atomic}_add_negative(${int} i, ${atomic}_t *v)
	{		{
	return ${atomic}_add_return(i, v) < 0;		return ${atomic}_add_return(i, v) < 0;

scripts/atomic/fallbacks/add_unless

+1 −1

Original line number	Original line	Diff line number	Diff line
	@@ -8,7 +8,7 @@ cat << EOF
	* Atomically adds @a to @v, if @v was not already @u.		* Atomically adds @a to @v, if @v was not already @u.
	* Returns true if the addition was done.		* Returns true if the addition was done.
	*/		*/
	static inline bool		static __always_inline bool
	${atomic}_add_unless(${atomic}_t *v, ${int} a, ${int} u)		${atomic}_add_unless(${atomic}_t *v, ${int} a, ${int} u)
	{		{
	return ${atomic}_fetch_add_unless(v, a, u) != u;		return ${atomic}_fetch_add_unless(v, a, u) != u;

scripts/atomic/fallbacks/andnot

+1 −1

Original line number	Original line	Diff line number	Diff line
	cat <<EOF		cat <<EOF
	static inline ${ret}		static __always_inline ${ret}
	${atomic}_${pfx}andnot${sfx}${order}(${int} i, ${atomic}_t *v)		${atomic}_${pfx}andnot${sfx}${order}(${int} i, ${atomic}_t *v)
	{		{
	${retstmt}${atomic}_${pfx}and${sfx}${order}(~i, v);		${retstmt}${atomic}_${pfx}and${sfx}${order}(~i, v);